LENS: LLM-Enabled Narrative Synthesis for Mental Health by Aligning Multimodal Sensing with Language Models
Researchers have introduced LENS, a novel framework designed to bridge the gap between multimodal health sensing data and large language models (LLMs) for mental health assessment. Addressing the challenge of translating numerical time-series sensor measurements into natural language, LENS aligns raw behavioral signals with LLMs to generate clinically grounded narratives. The team constructed a large-scale dataset comprising over 100,000 sensor-text question-answer pairs derived from Ecological Momentary Assessment responses of 258 participants focusing on depression and anxiety. A key technical innovation is a patch-level encoder that projects raw sensor signals directly into the LLM's representation space, enabling native time-series integration. Evaluation results demonstrate that LENS outperforms strong baselines in standard NLP metrics and symptom-severity accuracy. Furthermore, a user study involving 13 mental health professionals confirmed that the generated narratives are comprehensive and clinically meaningful. This approach advances the use of LLMs as interfaces for health sensing, offering a scalable solution for reasoning over raw behavioral data to support clinical decision-making processes.
Wire timeline
LENS: LLM-Enabled Narrative Synthesis for Mental Health by Aligning Multimodal Sensing with Language Models
Researchers have introduced LENS, a novel framework designed to bridge the gap between multimodal health sensing data and large language models (LLMs) for mental health assessment. Addressing the challenge of translating numerical time-series sensor measurements into natural language, LENS aligns raw behavioral signals with LLMs to generate clinically grounded narratives. The team constructed a large-scale dataset comprising over 100,000 sensor-text question-answer pairs derived from Ecological Momentary Assessment responses of 258 participants focusing on depression and anxiety. A key technical innovation is a patch-level encoder that projects raw sensor signals directly into the LLM's representation space, enabling native time-series integration. Evaluation results demonstrate that LENS outperforms strong baselines in standard NLP metrics and symptom-severity accuracy. Furthermore, a user study involving 13 mental health professionals confirmed that the generated narratives are comprehensive and clinically meaningful. This approach advances the use of LLMs as interfaces for health sensing, offering a scalable solution for reasoning over raw behavioral data to support clinical decision-making processes.
cs.AI updates on arXiv.org