A clinical classification for radiation-less monitoring of scoliosis based on deep learning of back photographs

Teng Zhang, Chuang Zhu, Yongkang Zhao, Moxin Zhao, Zhihao Wang, Ruoning Song, Nan Meng, Alisha Sial, Ashish D. Diwan, Jun Liu, Jason Pui Yin CheungResearch Square 2022 · DOI

TL;DR

Researchers have developed ScolioNet, a deep learning framework that classifies the severity and curve type of Adolescent Idiopathic Scoliosis (AIS) using only smartphone back photographs. By achieving high diagnostic accuracy (AUCs up to 0.902), this radiation-free approach offers a scalable way to monitor spinal progression outside of traditional clinical settings.

What Problem Does This Paper Address?

Adolescent Idiopathic Scoliosis (AIS) is a 3D spinal deformity affecting up to 4.8% of girls. Monitoring AIS is a long-term commitment; patients require frequent check-ups to decide if they need bracing or surgery. Currently, this relies on physical exams (which are subjective) and repeated X-rays.

The "gold standard" measurement, the Cobb Angle, requires radiography, but cumulative radiation exposure in children is linked to increased risks of breast cancer and heritable defects. Furthermore, the COVID-19 pandemic highlighted the fragility of hospital-based monitoring. The authors sought to bridge this gap by creating an objective, radiation-less, and accessible screening tool that works with arbitrary smartphone photos.

Methods

The authors developed a mobile platform called AlignProCARE, powered by a deep learning model named ScolioNet.

  • Architecture: The model uses a ResNet50 backbone, modified with residual attention blocks to help the network focus on specific anatomical distortions (like rib humps or waist asymmetry).
  • Multi-task Learning: ScolioNet was trained to perform two tasks simultaneously: classifying the severity of the deformity (Normal-mild, Moderate, or Severe) and identifying the curve type (Thoracic, Thoracolumbar/Lumbar, or Mixed).
  • Interpretability: To ensure the model wasn't a "black box," the team used Score-CAM to generate heatmaps. These heatmaps visualize which areas of the back the model prioritized when making a classification, allowing clinicians to verify that the AI was looking at clinically relevant deformities.
  • Data Augmentation: To handle the "noise" of real-world smartphone photos (varying angles, lighting, and backgrounds), they employed aggressive augmentation, including random rotations, affine transforms, and contrast adjustments.

Datasets

The study utilized a large, high-quality dataset of paired back photographs and radiographs:

  • Cohort 1 (Development): 1,780 patients (mean age 14) used for training and internal validation.
  • Cohort 2 (Independent Testing): 378 patients recruited prospectively to test the model’s performance in a real-world clinical flow.
  • Ground Truth: All images were manually annotated by experienced deformity surgeons based on X-ray data (Cobb Angles and apical vertebra locations).

Key Insights and Ideas

  • AI vs. Human Experts: In a head-to-head comparison, ScolioNet significantly outperformed both senior and junior surgeons who were asked to classify the same back photos. For example, the model's sensitivity for identifying patients needing follow-up was ~85%, while a senior surgeon’s was only ~44%. This suggests the CNN detects subtle topographic cues invisible to the human eye.
  • High Negative Predictive Value (NPV): The model achieved an NPV of approximately 90%. In a screening context, this is crucial; it means if the model says a child doesn't need surgery or a follow-up, there is a very high probability that the assessment is correct, safely reducing unnecessary X-rays.
  • Anatomical Alignment: The Score-CAM heatmaps showed that for thoracic curves, the model focused on the upper back/rib cage, whereas for lumbar curves, it focused on the lower back and waistline. This alignment with human clinical logic increases trust in the system's deployment.

Key Results

The model demonstrated robust performance across several metrics on the independent test set:

  • Severity Classification: Achieved an Area Under the Curve (AUC) of 0.839 for recommending follow-up and 0.902 for recommending surgery.
  • Curve Type Classification: AUCs were 0.777 (Thoracic), 0.760 (Thoracolumbar/Lumbar), and 0.860 (Mixed).
  • Clinical Utility: The system is deployed via a web and mobile app (AlignProCARE), allowing for "out-of-hospital" assessments that can be remotely reviewed and modified by clinicians.

Limitations and Future Work

  • Domain Shift: The authors noted a slight performance drop in the prospective cohort compared to the training set. This was attributed to "domain shift"—variations in camera position, lighting, and ambient environment that occur in real-world usage.
  • Data Imbalance: The dataset contained fewer "severe" cases compared to "moderate" ones, which is typical in clinical practice but can challenge model training.
  • Future Directions: The authors suggest that a multi-center trial is necessary to further validate the model's robustness across different populations and photography setups. They also aim to refine the model to better handle the "Mixed" curve types, which are more complex to distinguish than single curves.

Summary updated

Abstract

<title>Abstract</title> Routine check-ups for adolescent idiopathic scoliosis are critical to monitor progression and prescribe interventions. AIS is primarily screened via physical examination. If there are features of deformity, radiographs are necessary for diagnosis or follow-up, guiding further management, i.e., bracing for moderate deformity and surgery for severe. However, this subjects children to repetitive radiation and routine practices can be disturbed. Here, we demonstrate a mobile platform powered by <italic>ScolioNet</italic>, being trained and validated by 1780 back photos with paired radiographs, consisting of heterogeneous severities and curve types, manually annotated by experienced deformity surgeons. In further independent testing on 378 patients, the platform recommended follow-up or surgery with area under curves (AUCs) of receiver operating characteristic curves being 0.839, and 0.902, while distinguishing among thoracic, thoracolumbar/lumbar, or mixed curve types with AUCs of 0.777, 0.760 and 0.860 respectively. Our open platform has potential for out-of-hospital accessible applications in managing children with AIS.
BibTeX
@inproceedings{zhang2022clinical,
  author = {Teng Zhang and Chuang Zhu and Yongkang Zhao and Moxin Zhao and Zhihao Wang and Ruoning Song and Nan Meng and Alisha Sial and Ashish D. Diwan and Jun Liu and Jason Pui Yin Cheung},
  title = {{A clinical classification for radiation-less monitoring of scoliosis based on deep learning of back photographs}},
  booktitle = {Research Square},
  year = {2022},
  doi = {10.21203/rs.3.rs-1655808/v1},
}
Figure