TL;DR
This paper introduces the Discontinuation Prediction Score (DiPS), a machine learning model that accurately predicts future exercise relapse in individuals participating in physical activity programs. By leveraging objectively measured activity data, DiPS offers a promising tool for tailoring interventions and optimizing resource allocation to improve adherence.
What Problem Does This Paper Address?
Adherence to physical activity programs is a significant challenge, with many individuals struggling to maintain consistent engagement. This low adherence diminishes the health benefits of these programs and leads to costly relapses. While various factors influence adherence, predicting who is likely to relapse remains difficult, especially with real-time, objective data. Existing methods often rely on self-reported data or are not personalized enough. This paper aims to bridge this gap by developing and validating machine learning models that can predict future exercise relapse using continuous, objective physical activity data.
Methods
The researchers developed two versions of the Discontinuation Prediction Score (DiPS) using established machine learning techniques:
- Logistic Regression (LR): This method was chosen for its interpretability, allowing researchers to understand the importance of different features in predicting relapse.
- Support Vector Machine (SVM): SVM was used as a validation method to assess the robustness of the prediction approach to physical activity data.
Both models were trained on a comprehensive set of features extracted from accelerometer data, including daily steps, goal achievement percentages, and measures of physical activity intensity (METs) across different times of the day (morning, afternoon, evening). A key methodological innovation was the augmentation of the training data to create more observations per participant, allowing the models to learn from the temporal dynamics of activity. The DiPS score itself is designed to output a probability (between 0 and 1) indicating the likelihood of an individual having an exercise relapse in the following week.
Datasets
The study utilized data from the Mobile Phone-Based Physical Activity Education (mPED) trial. This dataset comprised 9 months of continuously collected, 60-second interval physical activity data from 210 community-dwelling, physically inactive women aged 25-69. The data included accelerometer readings for steps and activity intensity (METs). The study used the first 15 weeks of data for training and validation, and then tested the models on weeks 16-30.
Key Insights and Ideas
- Objective Data is Key: The study demonstrates that objectively measured physical activity data (steps, intensity) is highly predictive of future exercise relapse, outperforming reliance on sociodemographics or self-reported data alone.
- Predicting Relapse is Feasible: Machine learning models, specifically LR and SVM, can accurately predict future exercise relapse, opening doors for proactive interventions.
- Steps and Intensity Matter Most: The most significant predictors of relapse were found to be step counts (initial, average, and last week) and physical activity intensity. Interestingly, the time of day for physical activity was not a significant predictor.
- Initial Steps as a Baseline: Defining relapse relative to the initial run-in period's steps provides a meaningful benchmark for assessing progress and identifying deviations.
- Cost-Effective Interventions: Simulation results suggest that a DiPS-based approach to allocating financial incentives can be more cost-effective in improving adherence compared to random or simple step-threshold interventions.
Key Results
- High Prediction Accuracy: Both LR and SVM models achieved a high test Area Under the Curve (AUC) of approximately 0.90 for predicting exercise relapse.
- Robustness Across Weeks: The models demonstrated consistent performance across different weeks of the study, indicating their robustness.
- Sensitivity and Specificity: The DiPS model achieved an accuracy of 85% and a specificity of 67% in predicting relapse for a given week.
- Feature Importance: Week number, step-related features, and physical activity intensity were identified as the most predictive features.
- Simulation Efficacy: The DiPS-based intervention policy in simulations led to a higher percentage of adherent participants compared to random or step-based interventions, especially when a moderate number of interventions were applied.
Limitations and Future Work
The authors acknowledge several limitations:
- Population Specificity: The study data was collected from physically inactive female adults. Therefore, DiPS scoring needs to be tested and validated in other populations, including males and children.
- Time Horizon: The study's data collection extended only to 9 months. Predicting adherence beyond this period might be less accurate and requires further investigation.
Future work suggested includes:
- Validation in Larger RCTs: Conducting full-scale randomized controlled trials to validate DiPS efficacy in diverse populations and over longer periods.
- Exploring Other ML Methods: Evaluating the framework against other machine learning models, such as tree-based models.
- Real-World Implementation: Incorporating DiPS into mobile technology-based physical activity promotion programs to deliver automated, just-in-time interventions (e.g., push notifications, text messages) and personalized goal adjustments.
- Empirical Study for Simulation: Conducting an empirical study to validate the simulation findings regarding the cost-effectiveness of DiPS-based interventions.
Summary updated