InertiaMPDD: Multimodal Personalized Depression Detection via Emotional Inertia-Inspired Dynamics Modeling

Changzeng Fu, Songyang Wang, Chang Gao, Xinyi Zhang, Peilin Li, Mo Meng, Jiarui Wu, Mingze Li, Hongwei Ding, Shiqi Zhao, Yong Li, Lang He

TL;DR

InertiaMPDD is a novel multimodal framework that detects depression by comparing a person's behavior to a personalized "healthy reference" model. By measuring how much and how long an individual's emotional states deviate from this learned healthy baseline—a concept inspired by psychological "emotional inertia"—the system achieves state-of-the-art accuracy while providing time-resolved evidence of depressive traits.

What Problem Does This Paper Address?

Most current AI models for depression screening act as "black boxes" that output a single risk score for an entire recording. This approach has two major flaws:

  1. Lack of Personalization: Behavioral cues like slow speech or limited facial movement might be a sign of depression for one person but perfectly normal (habitual) for another. Without an explicit reference for what "normal" looks like for a specific context, population-level models can struggle.
  2. Coarse Supervision: Depression datasets usually provide one label per recording. This makes it difficult for models to identify exactly which segments of a long video are informative or how emotional states evolve over time.

The authors tackle these issues by drawing on affective science, specifically the concept of emotional inertia—the tendency for emotional states to persist longer than usual, which is a known hallmark of depression.

Methods

The researchers introduced InertiaMPDD, which uses a dual-pathway architecture to combine population-level patterns with individual behavioral departures.

Figure 1: Motivation. (a) Existing Multimodal Approaches. (b) Our Emotion Dynamics Trait Analysis.

1. Nomothetic Screening Pathway (NSP)

This is the "population" view. It uses a large multimodal backbone (Qwen2.5-Omni-3B) adapted via Low-Rank Adaptation (LoRA) to predict a general depression risk margin from audio, video, and text.

2. Idiographic Inertia Pathway (IIP)

This is the "individual" view and the paper's core innovation. Inspired by world models, the IIP learns to predict the "next state" of a healthy person given their recent history and current context.

  • Healthy Reference: The predictor is trained only on non-depressed individuals.
  • Dynamics Descriptors: By comparing the actual observed behavior to what the "healthy" model predicted, the system extracts three key metrics:
  • Reference Deviation ($R$): The average "disagreement" between the person and the healthy reference.
  • Residual Decay Rate ($force_{AR}$): How long these disagreements persist (measuring inertia).
  • Jacobian Spectral Descriptor ($\rho(J)$): A measure of the local stability of the emotional dynamics.

The final screening score is the sum of the population margin and an individual correction derived from these descriptors.

Figure 2: InertiaMPDD framework. (a) Input encoding. (b) NSP population margin and IIP healthy-reference descriptors. (c) Individual correction and screening. Snowflakes mark frozen parameters. Curves and representations are schematic.

Datasets

The model was evaluated across five cohorts from four diverse datasets:

  • D-Vlog: Spontaneous English YouTube vlogs.
  • LMVD: A large-scale Chinese vlog dataset.
  • AVEC2014: Clinical interview-based recordings (prompted responses and reading).
  • MPDD: A personality-aware dataset featuring both "elder" and "young" cohorts.

Key Insights and Ideas

  • Predictive Departure as a Biomarker: The "aha moment" is that we don't need fine-grained emotion labels to find depressive evidence. Instead, we can treat depression detection as an anomaly detection task relative to a "healthy" behavioral manifold.
  • Context Matters: The model uses "context conditioning" (mean and variance of features) to ensure the healthy reference is appropriate for the specific recording. This helps distinguish between a naturally quiet person and someone showing depressive withdrawal.
  • Inertia is Measurable: The study successfully translates the psychological theory of emotional inertia into a computable "residual decay rate," proving that the persistence of behavioral deviations is often more informative than the deviations themselves.
  • Setting-Dependent Utility: The personalized reference was highly effective for spontaneous vlogs and guided interactions (MPDD) but less so for highly constrained tasks like reading aloud (AVEC2014), suggesting that "healthy" behavior is easier to define when people have more freedom to express themselves.

Key Results

InertiaMPDD demonstrated significant performance gains over existing state-of-the-art methods:

  • D-Vlog: Achieved a staggering 98.41% F1 score.
  • LMVD: Reached 84.98% F1, a 9.10-point gain over the strongest single pathway.
  • MPDD: The standalone IIP descriptors (without the population model) reached 92.31% F1 on the elder cohort, outperforming standard fusion methods.
  • Visual Evidence: Case studies showed that while healthy individuals fluctuate around their reference, depressed individuals often show "sustained residual elevations"—their behavior stays consistently "off" from the healthy prediction for long periods.
Figure 5: Case studies showing how depressed traces (red) consistently deviate from the healthy reference (green) compared to healthy subjects.

Limitations and Future Work

  • Task Constraints: The model struggled on the AVEC2014 dataset. The authors suggest this is because the dataset's prompted tasks (like reading) might not provide enough behavioral variance for a "healthy reference" to be discriminative.
  • Reference Coverage: A "healthy" model is only as good as the data it's trained on. Future work needs to ensure the reference pool covers a wider range of ages, languages, and cultural contexts to avoid bias.
  • Specificity: While the model detects departures from "healthy," it's not yet clear if these departures are specific to depression or could also indicate other conditions like high stress or fatigue. Testing for specificity is a key next step.

Summary updated

Abstract

Multimodal depression screening estimates risk from speech, visual behavior and language. Current methods mainly predict labels, with limited evidence of departures from a non-depressed reference. Recording-level supervision also leaves fine-grained emotional dynamics unannotated. Motivated by emotional inertia, the persistence of affect, InertiaMPDD learns healthy next-state patterns using a world-model-inspired latent predictor. Using pretrained multimodal representations, it predicts from observed history and context. Reference deviation, residual decay rate and a Jacobian spectral descriptor form reference-derived emotional dynamics descriptors, evaluated as candidate proxy biomarkers for standalone screening and personalized decision correction. Across five cohorts from four datasets, fusion reaches 98.41% F1 on D-Vlog and 84.98% on LMVD. Standalone descriptors outperform fixed additive fusion on MPDD but discriminate weakly on AVEC2014. These findings distinguish reference informativeness from fusion benefit and reveal context-dependent utility.
BibTeX
@misc{fu2026inertiampdd,
  author = {Changzeng Fu and Songyang Wang and Chang Gao and Xinyi Zhang and Peilin Li and Mo Meng and Jiarui Wu and Mingze Li and Hongwei Ding and Shiqi Zhao and Yong Li and Lang He},
  title = {{InertiaMPDD: Multimodal Personalized Depression Detection via Emotional Inertia-Inspired Dynamics Modeling}},
  year = {2026},
  eprint = {2611.16647},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.16647},
}
Figure