OAK-Mind: Zero-Training Mental Health Change Inference from Passive Sensing via Knowledge-Grounded Multi-Agent LLMs

Yishu Ji, Zhihan Jiang, Will Ke Wang, Yuang Fan, Anind K. Dey, Xuhai "Orson" XuarXiv 2026 · arXiv

TL;DR

OAK-Mind is a zero-training pipeline that uses multi-agent Large Language Models (LLMs) to infer changes in mental health from passive sensing data (like sleep and step counts). By grounding specialized agents in clinical literature and structuring them to "reason" through behavioral changes, the system achieves performance comparable to or better than supervised machine learning models—all without requiring task-specific training data.

What Problem Does This Paper Address?

Passive sensing—using smartphones and wearables to monitor behavior—offers a continuous window into mental health. However, traditional approaches rely on supervised machine learning, which presents three major hurdles:

  1. Data Scarcity: High-quality, labeled mental health datasets are difficult and expensive to collect.
  2. Generalization: Models trained on one population often fail when applied to another.
  3. The "Black Box" Problem: Supervised models rarely provide a clinical rationale for their predictions, making them hard for practitioners to trust.

While LLMs offer reasoning capabilities, they aren't naturally structured to handle noisy, numerical time-series data from sensors. OAK-Mind addresses this by creating a structured, knowledge-grounded framework that allows LLMs to interpret behavioral shifts without any fine-tuning.

Methods

The OAK-Mind pipeline (shown in Figure 1) organizes inference into three distinct agentic layers:

  1. Behavioral Change Agents: Four specialized agents analyze longitudinal changes in specific domains: Sleep, Physical Activity, Mobility, and Phone Usage. Each agent is provided with "Domain Knowledge" (summarized in Figure 4)—clinical insights extracted from a literature-based knowledge graph—to help it interpret what specific sensor changes (e.g., increased sleep variability) mean for mental health.
  2. Cross-Domain Synthesis Agent: This agent integrates the narratives from the four behavioral agents into a single holistic assessment. It weighs conflicting evidence (e.g., improved mobility but worsening sleep) and considers the individual's baseline score to determine the likely trajectory.
  3. Symptom Rating Agent: Finally, this agent maps the synthesis onto a clinical framework. It "fills out" the PHQ-4 questionnaire (Table 1) based on the behavioral evidence, producing a final 3-class prediction (Improved, Stable, or Worsened) along with a human-readable rationale.
Fig. 1. The overall design of the OAK-Mind pipeline

Unlike standard LLM prompting, OAK-Mind uses a "zero-training" approach, meaning it uses off-the-shelf models (like GPT-4 or Gemini) without any task-specific fine-tuning or in-context examples from other users.

Datasets

The researchers evaluated OAK-Mind on two longitudinal datasets:

  • GLOBEM: A multi-year dataset with 497 participants, featuring wearable and mobile sensing data paired with weekly depression and anxiety assessments.
  • College Experience: A four-year study of 215 college students, focusing on mobile sensing data and weekly self-reports.

Key Insights and Ideas

  • Knowledge as Scaffolding: Simply giving an LLM sensor data isn't enough. By providing "reasoning scaffolds" derived from clinical literature (see Figure 4), the agents can better distinguish between healthy behavioral shifts and those indicative of symptom changes.
  • The Power of Decomposition: Ablation studies showed that using domain-specific agents (Sleep vs. Mobility) outperformed a single "Unified Behavioral Agent." This suggests that specialized roles prevent the LLM from being overwhelmed by multi-modal data and lead to more nuanced interpretations.
  • Clinical Grounding: By forcing the final agent to use a validated clinical tool (the PHQ-4), the system's output becomes "traceable." A clinician can see exactly which behavioral patterns led to a specific symptom rating.
Table 2. Domain knowledge embedded in each Behavioral Change Agent for the depression and anxiety tasks.

Key Results

OAK-Mind demonstrated that "zero-training" doesn't mean "low performance":

  • Outperforming Supervised Baselines: On the GLOBEM depression task, OAK-Mind achieved an accuracy of 0.56, significantly higher than the best supervised baseline (0.47), which was trained on hundreds of additional labeled samples.
  • Robustness Across Backbones: The system showed consistent performance gains over direct LLM prompting across four different model backbones: GPT-4, Qwen-plus, Gemini-1.5-flash, and DeepSeek-chat.
  • Interpretability: The pipeline produces detailed narratives for every step. For example, it might note that "reduced sleep disruptions and more consistent mobility patterns suggest a modest improvement in low mood," providing a clear justification for its "Improved" label.

Limitations and Future Work

The authors acknowledge several areas for improvement:

  • Population Bias: Both datasets consisted of college students. Future work needs to test OAK-Mind on more diverse demographic groups and clinical populations.
  • Brief Measures: The study used the PHQ-4, a very brief screening tool. Future iterations could target more comprehensive diagnostic assessments.
  • Narrative Faithfulness: While the rationales are human-readable, the study did not formally evaluate the "faithfulness" of these narratives (i.e., whether the LLM's written reasoning perfectly matches its internal logic).
  • Real-world Utility: Future research should investigate how clinicians or patients might actually use these generated narratives to support mental health management.

Reference: Ji, Y., Jiang, Z., Wang, W. K., Fan, Y., Dey, A. K., & Xu, X. "OAK-Mind: Zero-Training Mental Health Change Inference from Passive Sensing via Knowledge-Grounded Multi-Agent LLMs." arXiv:2611.22248v1, 2026.

Summary updated

Abstract

Wearable and smartphone sensors capture behavioral signals associated with mental health, yet inferring mental health outcomes from these signals has traditionally relied on supervised models requiring labeled data and offering limited insight into their predictions. Large language models (LLMs) offer new opportunities to reason over behavioral evidence, but existing LLM-based approaches to passive sensing have largely continued to rely on task-specific fine-tuning or sensor-specific pretraining. In this paper, we introduce OAK-Mind, a zerO-training, multi-Agent, Knowledge-grounded LLM pipeline for inferring mental health changes from passive sensing data. The pipeline organizes sensing data into behavioral dimensions including sleep, physical activity, location mobility, and smartphone use, and orchestrates specialized agents grounded in mental health literature to interpret longitudinal behavioral changes. It synthesizes their assessments and applies questionnaire-based symptom rating to infer changes relative to an individual's baseline assessment with human-readable rationale grounded in observed behavioral patterns and domain knowledge. Across datasets and prediction tasks, while without task-specific training, OAK-Mind achieves comparable or better performance than the most predictive supervised baselines trained on additional labeled samples. It also demonstrate consistent improvement upon direct prompting across four tested LLM backbones. Ablation analyses support the benefits of combining behavioral interpretation with symptom rating, distributing analysis across domain-specific agents, and incorporating domain knowledge. Our findings highlight the potential of knowledge-grounded, multi-agent LLMs for training-free inference of mental health changes from passive sensing data.
BibTeX
@misc{ji2026oak,
  author = {Yishu Ji and Zhihan Jiang and Will Ke Wang and Yuang Fan and Anind K. Dey and Xuhai "Orson" Xu},
  title = {{OAK-Mind: Zero-Training Mental Health Change Inference from Passive Sensing via Knowledge-Grounded Multi-Agent LLMs}},
  year = {2026},
  eprint = {2611.22248},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.22248},
}
Figure