Personalised depression forecasting using mobile sensor data and ecological momentary assessment

Alexander Kathan, Mathias Harrer, Ludwig Küster, Andreas Triantafyllopoulos, Xiangheng He, Manuel Milling, Maurice Gerczuk, Tianhao Yan, Srividya Tirunellai Rajamani, Elena Heber, Inga Grossmann, David D. Ebert, Björn W. SchullerFrontiers Digit. Health 2022 · DOI

TL;DR

This paper explores personalized machine learning models for predicting daily depressive symptom severity using mobile sensor data and ecological momentary assessments (EMA). They found that personalized approaches, particularly subject-dependent standardization, significantly improve prediction accuracy and fairness compared to general models, offering a promising avenue for more effective digital mental health interventions.

What Problem Does This Paper Address?

Depression is a widespread and debilitating condition, and while digital interventions show promise, they often suffer from a "one-size-fits-all" approach. This leads to varying treatment effectiveness, with many patients not achieving remission. Understanding how individual symptoms evolve dynamically is crucial for personalizing treatments. Current forecasting models are typically trained on population-level data, which can overlook individual differences and lead to biased predictions, especially in clinically heterogeneous populations. This paper aims to address this by investigating personalized machine learning models that can adapt to individual patient trajectories, improving both prediction accuracy and fairness.

Methods

The researchers investigated three personalization strategies:

  • Transfer Learning with Shared Common Layers: This approach uses a common "backbone" model trained on all participants, with personalized "layers" fine-tuned for each individual. This allows the model to learn general patterns while adapting to individual nuances.
  • Personalized Subgroup Models: Here, models are trained separately for distinct subgroups. In this study, they used gender as the subgroup criterion, creating separate models for male and female participants.
  • Personalization with Subject-Dependent Standardization: This method uses a single common model but applies standardization of the data separately for each subject. This means the model learns general patterns, but the way the data is scaled is personalized.

They evaluated these methods using a 3-fold cross-validation approach, focusing on predicting the Patient Health Questionnaire-2 (PHQ-2) score at the end of the day and forecasting it for the next day. Mean Absolute Error (MAE) and Spearman's ρ correlation were used as evaluation metrics.

Datasets

The study utilized a newly collected, longitudinal dataset called the "MAIKI" (Mobile Therapy Assistant for Daily Life with Interaction-focused Artificial Intelligence for Depression) dataset. This dataset comprises data from 65 patients undergoing a digital depression intervention. It includes:

  • Active Data:
    • Ecological Momentary Assessments (EMAs): Daily self-reports on mood, affect, and other symptoms.
  • Daily diary entries.
  • Weekly symptom screenings (PHQ-9, GAD-7).
  • Baseline assessments (sociodemographics, clinical interviews).
  • Passive Mobile Sensor Data:
    • GPS data (location variance, distance traveled, time at home).
  • Communication data (call frequency, duration, missed calls).
  • Phone usage data (frequency, duration).
  • User activity data (app usage, lock screen duration).
  • Sleep behavior data.

Key Insights and Ideas

  • Personalization is Key for Depression Forecasting: The core insight is that tailoring models to individual patients significantly boosts their predictive power for depressive symptoms, moving beyond generic population-level models.
  • Subject-Dependent Standardization Shines for Prediction: This specific personalization technique emerged as the most effective for predicting daily PHQ-2 scores, demonstrating a substantial improvement over the baseline.
  • Transfer Learning Excels in Forecasting: For predicting symptoms one day ahead, the transfer learning approach proved most beneficial, showing a notable improvement in forecasting accuracy.
  • Personalization Improves Fairness: Beyond accuracy, personalized models were found to be fairer, reducing biases related to sex that were present in the non-personalized baseline model. This is critical for equitable digital health tools.
  • EMA Data is Highly Informative: The study highlighted the significant contribution of EMA data to prediction accuracy, underscoring its value in capturing real-time symptom fluctuations.

Key Results

  • Prediction: Subject-dependent standardization achieved the best Mean Absolute Error (MAE) of 0.801 for predicting daily PHQ-2 scores, a 25% improvement over the non-personalized baseline (MAE = 1.062).
  • Forecasting: The transfer learning approach yielded the best MAE of 1.349 for one-day-ahead forecasting, a 12% improvement over the baseline (MAE = 1.539).
  • Fairness: Personalization methods generally reduced sex-based biases observed in the baseline model. Subgroup models nearly eliminated bias for prediction, while transfer learning was best for forecasting.
  • Individual-Level Fairness: While personalization increased the divergence between "winners" (those with better performance) and "losers" (those with worse performance), it generally led to improved MAE scores across the board.

Limitations and Future Work

  • Data Availability and Quality: A significant limitation was the presence of missing data, which reduced the number of participants eligible for analysis. This is a common challenge with real-world mobile data. Future work could explore more robust missing data imputation techniques or methods that are less sensitive to data sparsity.
  • "Burn-in" Period: Some personalized models require several weeks of individual training data, which can delay the availability of forecasts in clinical practice. Future research could investigate ways to shorten this "burn-in" phase or use hybrid approaches that start with general models and transition to personalized ones.
  • Overfitting: The risk of overfitting personalized models to limited individual data was acknowledged. The paper suggests using population-level features (transfer learning) or subgroup models as ways to mitigate this.
  • Feature Importance: The study found that actively assessed data (like EMAs) were highly important for prediction. Future work could focus more on these data types to potentially scale up personalized forecasting with lower technical requirements.
  • Generalizability: While personalization improves individual predictions, ensuring generalizability across diverse populations and contexts remains an ongoing challenge.

The authors conclude that personalization, particularly with subject-dependent standardization and transfer learning, shows significant promise for improving depression prediction and forecasting, as well as enhancing fairness in digital mental health applications. Further research with larger samples and exploration of more sophisticated personalization strategies is warranted.

Summary updated

BibTeX
@article{kathan2022personalised,
  author = {Alexander Kathan and Mathias Harrer and Ludwig Küster and Andreas Triantafyllopoulos and Xiangheng He and Manuel Milling and Maurice Gerczuk and Tianhao Yan and Srividya Tirunellai Rajamani and Elena Heber and Inga Grossmann and David D. Ebert and Björn W. Schuller},
  title = {{Personalised depression forecasting using mobile sensor data and ecological momentary assessment}},
  journal = {Frontiers Digit. Health},
  year = {2022},
  doi = {10.3389/fdgth.2022.964582},
}
Figure