TL;DR
This paper introduces a novel, open-source pipeline for early cerebral palsy (CP) risk assessment using infant videos. By leveraging off-the-shelf pose estimation and automated machine learning, the pipeline achieves moderate predictive accuracy and is designed for broad applicability across datasets, paving the way for more accessible global screening tools.
What Problem Does This Paper Address?
Cerebral palsy (CP) is a common motor disorder affecting movement control, with early detection and intervention being crucial for improved outcomes. The General Movements Assessment (GMA), performed on infant videos at 3-4 months, is a highly predictive tool for CP risk. However, GMA relies on trained clinicians, creating a bottleneck. While machine learning (ML) shows promise for automating GMA scoring from videos, existing approaches often suffer from dataset-specific preprocessing, custom feature engineering, and a lack of open-source implementations. This makes it difficult to benchmark models across different datasets, assess their real-world clinical utility, and train on larger, more diverse datasets, which is essential given the low prevalence of CP. The paper aims to bridge this gap by developing a generalizable, open, and reproducible pipeline for early CP risk assessment.
Methods
The core of the paper is an end-to-end pipeline designed for reproducibility and broad applicability. Key components include:
- Pose Estimation: Instead of custom fine-tuning, the authors utilized a pre-trained vision transformer model, ViTPose-H, known for its strong performance on diverse datasets without requiring dataset-specific adjustments. This addresses a major hurdle in applying ML to infant videos, where body poses can be challenging.
- Feature Extraction: A set of 38 general-purpose kinematic features (e.g., displacement, velocity, acceleration, entropy) were extracted from the pose estimates. These features were selected based on clinician input and are not specific to GMA's "fidgety movements" (FMs), allowing for broader application.
- Automated Machine Learning (AutoML): To minimize bias from hyperparameter tuning and model selection, the authors employed Auto-sklearn 2.0. This framework systematically explores different ML algorithms and settings, reducing the risk of overfitting and improving generalizability.
- Preregistered Study Design: A crucial aspect is the preregistration of the entire analysis pipeline, including feature selection, preprocessing, and model training, before accessing the "lock-box" test set. This strict protocol enhances transparency and reduces the likelihood of optimistic bias in performance reporting.
Datasets
- Primary Dataset: A newly generated dataset of 1,053 infant videos collected from the Children's Hospital of Philadelphia (CHOP) as part of standard clinical care. This dataset includes clinician-assessed GMA scores and demographic information. Approximately 10-12% of infants in this cohort were classified as having adverse GMA outcomes (FM-).
- Pose-Estimation Evaluation Datasets: For selecting the pose estimation algorithm, the authors used a combination of datasets: 94 videos from the YouTube-8M dataset, 76 videos from the CHOP Infant Dataset's training subset, and 130 additional infant videos from the "PANDA Gym" project.
Key Insights and Ideas
- Generalizability over Specificity: The paper strongly advocates for using off-the-shelf, pre-trained models and general features rather than dataset-specific fine-tuning. This approach is key to enabling collaboration and building larger, more robust models.
- The Power of Aggregation: While individual kinematic features showed little discriminative power, aggregating them into a feature vector significantly improved predictive performance. This highlights that subtle patterns emerge when considering multiple movement characteristics.
- Preregistration as a Standard: The commitment to a preregistered, "lock-box" evaluation strategy is a significant contribution, setting a high bar for transparency and reproducibility in this field.
- Open Science is Crucial: By releasing de-identified feature data and open-source code, the authors aim to accelerate research and facilitate the development of globally relevant tools.
Key Results
- Predictive Accuracy: The developed pipeline achieved a moderate predictive accuracy for clinician-assessed GMA scores on the preregistered lock-box test set:
- ROC-AUC: 0.77
- PR-AUC: 0.41 (noteworthy given the ~10-12% positive class prevalence)
- Robustness: Five-fold cross-validation on the training/validation sets yielded an average ROC-AUC of 0.73 ± 0.05, indicating good internal generalizability.
- Scaling: A scaling analysis demonstrated that model performance (ROC-AUC) improves with increasing dataset size, following a power-law relationship, underscoring the benefit of multi-site data aggregation.
- Feature Importance: Permutation importance analysis revealed that the model utilizes a combination of all features, with knee and elbow features contributing most strongly, followed by wrists and ankles. No single feature was a strong predictor on its own.
Limitations and Future Work
- Predicting GMA vs. CP Diagnosis: The study predicted GMA scores, a strong indicator of CP risk, rather than a definitive CP diagnosis. This introduces potential noise from human assessment error and limits the prediction window to 3-4 months. Future work should aim to predict CP outcomes directly.
- Subtle Movement Patterns: The current general kinematic features might miss subtle movements that are clinically relevant for early detection, especially before the typical 3-4 month window. More specialized feature extraction, potentially incorporating FM-specific features, could improve performance.
- General Population Screening: The current dataset primarily consists of high-risk infants. Future work should include infants from the general population to develop a broader prescreening tool.
- Video Quality and Diversity: The pipeline's robustness to variations in video quality, diverse infant skin tones, and lighting conditions needs further validation.
- CP Subtypes: The current model predicts overall risk. Future research should focus on differentiating CP subtypes, which may manifest as unique movement patterns.
- Threshold Selection: The paper acknowledges that threshold selection for classification was not part of the preregistered plan and was done for illustrative purposes. Future work will include threshold selection in prospective analysis plans.
Summary updated