Personalized Longitudinal Assessment of Multiple Sclerosis Using Smartphones

Oliver Y. Chén, Florian Lipsmeier, Huy Phan, Frank Dondelinger, Andrew P. Creagh, Christian Gossens, Michael Lindemann, Maarten De VosIEEE JBHI 2023 · OpenReview

TL;DR

This paper introduces a novel framework for personalized, longitudinal assessment of Multiple Sclerosis (MS) using smartphone sensor data. By combining multiple imputation, generalized estimation equations (GEE), and subject-specific fine-tuning, the model accurately predicts MS disease severity over time, even with missing data, offering a more accessible and frequent monitoring solution.

What Problem Does This Paper Address?

Managing Multiple Sclerosis (MS) effectively requires continuous monitoring of disease progression and individual patient trajectories. Traditional clinical assessments are infrequent, making it difficult to adapt treatment strategies in a timely manner. While smartphone sensors offer a promising avenue for frequent, remote data collection, two major challenges arise: handling missing data in longitudinal sensor streams and developing models that can personalize predictions to individual patients, rather than just relying on population-level trends. This paper aims to bridge this gap by creating a robust, personalized longitudinal assessment tool using smartphone data.

Methods

The proposed framework employs a multi-stage approach:

  • Data Collection: Smartphone sensors capture digital measurements of gait and balance (via the Two-Minute Walk Test - 2MWT) and upper extremity function (via the Draw a Shape - DaS test) from MS patients.
  • Missing Data Imputation: Multiple Imputation (MI) is used to address missing values in the longitudinal sensor data. The authors explored different numbers of imputations, finding that around 15 imputations provided a good balance between accuracy and computational efficiency.
  • Longitudinal Modeling with GEE: Generalized Estimation Equations (GEE) are used to model the longitudinal disease trajectories. GEE is chosen for its ability to handle correlated longitudinal measurements and its robustness to mis-specified correlation structures.
  • Feature Selection: A two-step feature selection process is employed. First, features significantly associated with the outcome are identified. Second, GEE is used to select the most predictive features, ultimately constraining the model to six key features (four smartphone-based and two covariates like age and sex) to balance model complexity with the availability of complete data in the test set.
  • Ensemble Learning: Parameters learned from multiple imputed datasets are ensembled to create a unified predictive model. This approach helps to account for the uncertainty introduced by imputation.
  • Subject-Specific Fine-Tuning: To further personalize predictions and mitigate potential underestimation for individuals with severe disease, a subject-specific fine-tuning step is introduced. This adjustment uses data from the first day of observation to fine-tune the population-level predictions, effectively accounting for individual baseline characteristics.

Datasets

The study utilized data from a 24-week prospective study (clinicaltrials.gov identifier: NCT02952911) involving 76 MS patients. This dataset included:

  • Smartphone Sensor Data: Collected via daily assessments of upper extremity function, gait and balance, and cognition. Specifically, data from the Two-Minute Walk Test (2MWT) and Draw a Shape (DaS) test were used.
  • Clinical Outcome Data: Clinician-rated Expanded Disability Status Scale (EDSS) scores and self-reported Multiple Sclerosis Impact Scale (MSIS-29) scores. The primary focus was on predicting MSIS-29 scores due to their more frequent collection.

Key Insights and Ideas

  • Smartphone Sensors as Viable MS Markers: The study demonstrates that objective measurements from smartphone sensors, particularly those related to gait, balance, and upper extremity function, can serve as valuable digital markers for longitudinal MS assessment.
  • The Power of Personalization: Simply applying population-level models to new patients can lead to underestimation of disease severity. The proposed subject-specific fine-tuning is crucial for capturing individual disease trajectories.
  • Balancing Extrapolation and Personalization: The framework effectively balances the need to identify generalizable markers (extrapolation) with the necessity of accounting for individual patient differences (personalization).
  • MI is Crucial for Longitudinal Data: Effectively handling missing data through multiple imputation significantly improves the accuracy and completeness of longitudinal disease assessment.
  • Feature Importance: Specific features from the DaS test (e.g., "Figure-of-8 GOF," "Figure-of-8 AV," "Figure-of-8 drawing error") and the 2MWT ("Step power") were consistently identified as important predictors of MS disease severity.

Key Results

  • The proposed model achieved a significant correlation (r = 0.80, p < 0.001) between estimated and observed MSIS-29 scores in novel individuals.
  • The subject-specific fine-tuning further improved prediction accuracy, with an average correlation of r = 0.81 in bootstrap experiments.
  • The model demonstrated the ability to capture longitudinal disease trends, with predicted score changes generally aligning with observed categories of improvement, stability, or worsening.
  • GEE outperformed mixed-effect models and was slightly better than GLM for both mean and longitudinal assessment of MSIS-29 scores.

Limitations and Future Work

  • Short Study Period and Sample Size: The study acknowledges that the relatively short duration of data collection and the sample size might limit the ability to detect statistically significant longitudinal disease trends, particularly for individuals with stable disease.
  • Mild Impairment Focus: The study primarily included patients with mild MS-related functional impairment. Future work should explore the model's performance in broader populations, including those with severe or fluctuating disease profiles.
  • Limited Feature Modalities: While gait and upper extremity features were explored, the study suggests that incorporating other modalities like dexterity tests, cognitive data, or passively collected digital data could further enhance prediction accuracy.
  • Future Directions: The authors propose extending the framework to other MS-related clinical outcomes, evaluating its utility in MS prognosis, and exploring its application to other neurodegenerative diseases. Further research into more sophisticated personalization techniques, such as using longer test data for fine-tuning or extracting subject-level priors, is also suggested.

Summary updated

Abstract

Personalized longitudinal disease assessment is central to quickly diagnosing, appropriately managing, and optimally adapting the therapeutic strategy of multiple sclerosis (MS). It is also important for identifying idiosyncratic subject-specific disease profiles. Here, we design a novel longitudinal model to map individual disease trajectories in an automated way using smartphone sensor data that may contain missing values. First, we collect digital measurements related to gait and balance, and upper extremity functions using sensor-based assessments administered on a smartphone. Next, we treat missing data via imputation. We then discover potential markers of MS by employing a generalized estimation equation. Subsequently, parameters learned from multiple training datasets are ensembled to form a simple, unified longitudinal predictive model to forecast MS over time in previously unseen people with MS. To mitigate potential underestimation for individuals with severe disease scores, the final model incorporates additional subject-specific fine-tuning using data from the first day. The results show that the proposed model is promising to achieve personalized longitudinal MS assessment; they also suggest that features related to gait and balance as well as upper extremity function, remotely collected from sensor-based assessments, may be useful digital markers for predicting MS over time.
BibTeX
@article{chen2023personalized,
  author = {Oliver Y. Chén and Florian Lipsmeier and Huy Phan and Frank Dondelinger and Andrew P. Creagh and Christian Gossens and Michael Lindemann and Maarten De Vos},
  title = {{Personalized Longitudinal Assessment of Multiple Sclerosis Using Smartphones}},
  journal = {IEEE J. Biomed. Health Informatics},
  year = {2023},
  url = {https://openreview.net/forum?id=DqbYeWjic1},
}
Figure