Interpretable Deep Learning for the Remote Characterisation of Ambulation in Multiple Sclerosis using Smartphones

Andrew P. Creagh, Florian Lipsmeier, Michael Lindemann, Maarten De VosarXiv 2021 · arXiv

Introduction

Multiple Sclerosis (MS) is a complex and fluctuating neurological condition. Traditional clinical assessments, such as the Expanded Disability Status Scale (EDSS), are often infrequent and subjective, potentially missing subtle changes in a patient’s condition. While smartphone-based monitoring offers a path toward continuous, remote assessment via inertial sensors, two significant hurdles remain: the "small data" problem, where clinical datasets are often too sparse for deep learning, and the "black box" problem, where model decisions lack transparency for clinical practitioners.

This research demonstrates that Transfer Learning (TL) can significantly boost the performance of deep learning models in detecting MS gait impairments from smartphone data, even with limited clinical samples. Furthermore, by employing Layer-Wise Relevance Propagation (LRP), the study "opens the black box," showing that these models make decisions based on clinically relevant features like cadence and gait speed.

Methodology: Bridging Domains with Transfer Learning

The authors proposed a framework that leverages general human movement data to improve disease-specific classification. Instead of training a Deep Convolutional Neural Network (DCNN) from scratch on limited clinical data, they utilized a two-stage approach involving source domain pre-training and target domain fine-tuning.

Schematic of the proposed transfer learning and interpretability framework.

Data Sources and Training

The study utilized three primary datasets:

  • Source Domains (Pre-training): Massive open-source Human Activity Recognition (HAR) datasets, including UCI HAR (30 subjects) and WISDM (51 subjects), were used to teach the model fundamental movement patterns.
  • Target Domain (Fine-tuning): The FLOODLIGHT PoC dataset, containing data from 97 participants (24 Healthy Controls, 52 with mild MS, and 21 with moderate MS) performing Two-Minute Walk Tests (2MWT) via smartphones.

Model Architecture and Interpretability

The core insight of this work is that low-level features of walking—rhythm, impact, and balance—learned from healthy populations are highly transferable to disease-specific gait analysis. To ensure clinical trust, the authors used Layer-Wise Relevance Propagation (LRP). This technique back-propagates the model's decision to the original input signal, creating "relevance heatmaps" that highlight which specific parts of a walking bout (e.g., a specific step or tremor) influenced the classification.

Key Results and Clinical Validation

The Transfer Learning DCNN significantly outperformed traditional Support Vector Machines (SVM) and end-to-end DCNNs. In binary classification tasks, the TL approach showed an 8–15% improvement. In the more challenging multi-class task (Healthy vs. Mild MS vs. Moderate MS), the TL model achieved a 33% improvement over baseline methods.

LRP attribution for a patient with MS, highlighting clinically relevant gait features and model predictions.

As shown in the figure above, the model's decision-making process can be visualized. By combining LRP with Continuous Wavelet Transform (CWT) frequency analysis, the researchers observed that the model picks up on "signal perturbations"—micro-stumbles or irregularities characteristic of MS. LRP heatmaps confirmed that the models focus on cadence-based measures and gait speed, which are established clinical indicators of MS progression.

Discussion and Future Directions

While the results are promising, several limitations remain. The dataset is still relatively small for deep learning standards, and demographic imbalances between groups could introduce bias. Furthermore, the LRP analysis remains largely qualitative. Future research should aim to quantitatively validate these relevance scores against gold-standard biomechanical measurements.

The authors also suggest that moving beyond DCNNs to architectures like Recurrent Neural Networks (LSTMs) might better capture the temporal "flow" of gait. Nonetheless, this work represents a significant step toward making deep learning a reliable and interpretable tool for remote clinical monitoring in chronic neurological diseases.

Summary updated

Abstract

The emergence of digital technologies such as smartphones in healthcare applications have demonstrated the possibility of developing rich, continuous, and objective measures of multiple sclerosis (MS) disability that can be administered remotely and out-of-clinic. In this work, deep convolutional neural networks (DCNN) applied to smartphone inertial sensor data were shown to better distinguish healthy from MS participant ambulation, compared to standard Support Vector Machine (SVM) feature-based methodologies. To overcome the typical limitations associated with remotely generated health data, such as low subject numbers, sparsity, and heterogeneous data, a transfer learning (TL) model from similar large open-source datasets was proposed. Our TL framework utilised the ambulatory information learned on Human Activity Recognition (HAR) tasks collected from similar smartphone-based sensor data. A lack of transparency of "black-box" deep networks remains one of the largest stumbling blocks to the wider acceptance of deep learning for clinical applications. Ensuing work therefore aimed to visualise DCNN decisions attributed by relevance heatmaps using Layer-Wise Relevance Propagation (LRP). Through the LRP framework, the patterns captured from smartphone-based inertial sensor data that were reflective of those who are healthy versus persons with MS (PwMS) could begin to be established and understood. Interpretations suggested that cadence-based measures, gait speed, and ambulation-related signal perturbations were distinct characteristics that distinguished MS disability from healthy participants. Robust and interpretable outcomes, generated from high-frequency out-of-clinic assessments, could greatly augment the current in-clinic assessment picture for PwMS, to inform better disease management techniques, and enable the development of better therapeutic interventions.
BibTeX
@misc{creagh2021interpretable,
  author = {Andrew P. Creagh and Florian Lipsmeier and Michael Lindemann and Maarten De Vos},
  title = {{Interpretable Deep Learning for the Remote Characterisation of Ambulation in Multiple Sclerosis using Smartphones}},
  year = {2021},
  eprint = {2103.09171},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2103.09171},
}
Figure