A machine-learning based objective measure for ALS disease severity

Fernando Vieira, Subhashini Venugopalan, Alan S. Premasiri, Maeve McNally, Aren Jansen, Kevin McCloskey, Michael P. Brenner, Steven PerrinNpj Digit. Medicine 2022 · DOI

TL;DR

Researchers have developed a machine learning (ML) system that uses voice and accelerometer data to objectively measure ALS disease severity, offering a more granular and continuous assessment than the current subjective, questionnaire-based ALSFRS-R. This approach holds promise for more effective clinical trials and personalized monitoring of ALS progression.

What Problem Does This Paper Address?

Amyotrophic Lateral Sclerosis (ALS) is a progressive neurodegenerative disease that affects motor neurons. Accurately measuring its severity is crucial for evaluating the effectiveness of new treatments in clinical trials and for understanding disease progression. The current standard, the ALS Functional Rating Scale-Revised (ALSFRS-R), relies on subjective questionnaires completed by patients or clinicians. This subjectivity can lead to variability and a lack of precision, especially in capturing subtle, gradual changes in disease progression. Furthermore, the heterogeneity of ALS presentations makes it challenging to design efficient clinical trials that can detect therapeutic signals. This paper aims to bridge this gap by creating an objective, data-driven measure of ALS disease severity.

Methods

The core of this research lies in applying machine learning to physiological data to predict ALSFRS-R scores. The authors developed two main types of ML models:

  • Voice Model: A Convolutional Neural Network (CNN) was trained on audio recordings of patients speaking a specific sentence. The model takes spectrograms of the speech as input and predicts the probability distribution over the 5 ALSFRS-R score classes for the speech (bulbar) function.
  • Accelerometer Models: Various ML models, including linear regression, logistic regression, and Multi-Layer Perceptrons (MLPs), were trained on accelerometer data collected during prescribed limb-based exercises. These models predict ALSFRS-R scores for 9 functions (6 limb-related and 3 respiratory). The best-performing accelerometer model was an MLP using Fast Fourier Transform (FFT) features from 1 Hz data.

Both models were trained to predict the probability distribution of ALSFRS-R scores, allowing for a more nuanced output than the integer scores of the traditional ALSFRS-R.

Datasets

The study utilized data from the ALS Therapy Development Institute's (ALS-TDI) Precision Medicine Program (PMP).

  • Training and Validation Data: Data from 584 individuals with ALS were collected over approximately four years. This included voice recordings and accelerometer measurements, along with corresponding ALSFRS-R scores recorded within 60 days of the physiological measurements.
  • Test Set: A subset of participants was designated for the test set, including 54 individuals who received edaravone as part of their usual care. This cohort was used to evaluate the real-world effectiveness of the ML-derived measures in assessing drug effects.

Key Insights and Ideas

  • Objective Measurement is Feasible: The study demonstrates that objective measures derived from voice and accelerometer data can reliably predict ALSFRS-R scores, offering a powerful alternative to subjective assessments.
  • Granular Progression Tracking: The ML-derived continuous scores can capture gradual changes in disease severity that are missed by the integer-based ALSFRS-R, providing a more sensitive tool for monitoring progression.
  • Preservation of Correlation Patterns: The ML models successfully preserved the known correlations between different ALSFRS-R subscores (e.g., between speech, salivation, and swallowing), indicating that the models are learning meaningful relationships within the data.
  • Voice is a Strong Predictor for Bulbar Function: The voice model achieved a high AUC for predicting speech ALSFRS-R scores, highlighting the utility of speech analysis for assessing bulbar dysfunction.
  • Accelerometer Models Show Promise for Limb Function: While less accurate than the voice model for speech, the accelerometer models demonstrated good performance in predicting limb-related ALSFRS-R scores.
  • Respiratory Function Remains a Challenge: Both voice and accelerometer models showed weaker correlations with respiratory function ALSFRS-R scores, suggesting this area requires further investigation.

Key Results

  • Voice Model Performance: Achieved a multiclass AUC of 0.86 (95% CI, 0.85-0.88) for predicting speech ALSFRS-R scores on the test set.
  • Accelerometer Model Performance: Achieved a median multiclass AUC of 0.73 for predicting 6 limb-related ALSFRS-R functions. The best individual limb function AUC was 0.756 for walking.
  • Correlation with Ground Truth: ML-predicted ALSFRS-R scores showed strong correlations with ground-truth self-reported scores, with R² values ranging from 0.49 to 0.75 for speech and limb functions.
  • Edaravone Study: Retrospective analysis of 54 participants treated with edaravone showed no observable cohort-level changes in progression using either ML-predicted or self-reported ALSFRS-R scores, but individual participant plots highlighted the continuous ML-derived score's ability to show gradual changes.

Limitations and Future Work

  • Respiratory Function: The models showed limited success in predicting respiratory function ALSFRS-R scores, indicating a need for further development in this area.
  • Speech Phrase Optimization: The chosen speech phrase was not optimized for signal maximization, suggesting that using a battery of phrases developed by speech pathologists could improve bulbar symptom assessment.
  • Accelerometer Exercise Optimization: The prescribed movements for accelerometer data collection were not optimized for fine motor function, and future work could explore protocols that better capture this aspect of ALS.
  • Sensory Measures: The study did not include direct or surrogate sensory measures of respiratory function, which could be valuable for future model development.
  • Edaravone Cohort Size: The retrospective analysis of edaravone effectiveness was performed on a small cohort, limiting definitive conclusions about drug efficacy.

Future work could involve exploring more advanced ML architectures, optimizing data collection protocols, and integrating additional physiological measures to create a more comprehensive and robust objective assessment of ALS disease severity.

Summary updated

BibTeX
@article{vieira2022machine,
  author = {Fernando Vieira and Subhashini Venugopalan and Alan S. Premasiri and Maeve McNally and Aren Jansen and Kevin McCloskey and Michael P. Brenner and Steven Perrin},
  title = {{A machine-learning based objective measure for ALS disease severity}},
  journal = {Npj Digit. Medicine},
  year = {2022},
  doi = {10.1038/s41746-022-00588-8},
}
Figure