TL;DR
This paper introduces a novel, multi-dimensional deep learning approach for evaluating scoliosis using only mobile phone photos of a patient's back. It aims to overcome the limitations of traditional X-ray examinations by enabling convenient, at-home screening and dynamic monitoring, offering a more accessible and radiation-free alternative.
What Problem Does This Paper Address?
Scoliosis, a spinal curvature, is increasingly common in adolescents. Early detection and intervention are crucial to prevent severe deformities. Current gold-standard diagnostic methods, like X-rays, are not ideal for large-scale screening or continuous monitoring due to radiation exposure, cost, and inflexibility. While advanced imaging techniques exist, they often require professional equipment and are not suitable for home use. This paper addresses the need for a non-invasive, accessible, and accurate method for evaluating scoliosis severity and type using readily available mobile phone cameras, facilitating both initial screening and long-term rehabilitation tracking.
Methods
The proposed method integrates three key deep learning-based algorithms:
- Spinal Coronal Curvature Type Discrimination: This algorithm leverages YOLOv8 for keypoint detection on back images. It then uses curve fitting (specifically, least squares fitting) to model the spine's shape and identifies "C", "S", or multi-curve types based on the number of tangent points detected using a sliding window approach.
- Coronal Plane Deviation Evaluation: This involves two sub-methods:
- Cobb Angle Measurement: Based on the fitted spine curve, this method calculates the Cobb angle, the standard measure of scoliosis severity. It adapts the calculation for "C", "S", and multi-curve types.
- "Scapular Spine Triangle" Difference: This method forms a triangle using the two scapular points and the spine's endpoint. The absolute difference between the angles at the base of this triangle is used to quantify coronal plane bending.
- Trunk Rotation (ATR Angle) Measurement: This algorithm focuses on the sagittal plane. It involves:
- Salient Object Segmentation: Using SAM (Segment Anything Model) to isolate the human back from the background.
- Contour Extraction: Employing edge detection operators (Roberts, Prewitt, LOG) to outline the back. Prewitt is found to be most suitable.
- Trunk Rotation Calculation: Utilizing the Automatic Multi-scale Peak Detection (AMPD) algorithm to identify key points (peaks) on the back contour, specifically the protruding scapular points. The angle between these points and the horizontal line is then calculated to determine trunk rotation.
A fusion method is also proposed, combining the "Scapular Spine Triangle" difference for "C" type scoliosis and the Cobb angle measurement for "S" and multi-curve types to maximize accuracy.
Datasets
- Training and Testing: A dataset of 3640 standing half-length back photos, manually annotated by professional doctors, was used. This dataset was partitioned into training, verification, and test sets.
- Clinical Matched Data: 25 patient back photos paired with their corresponding X-rays were collected from a hospital.
- Trunk Rotation Dataset: 50 high-quality 90-degree backward bending photos were used.
Key Insights and Ideas
- Leveraging Mobile Photos: The core innovation is demonstrating that high-quality scoliosis assessment can be achieved using readily available mobile phone cameras, democratizing access to screening.
- Multi-Dimensional Approach: Moving beyond just coronal plane assessment, the paper incorporates sagittal plane (trunk rotation) evaluation, offering a more comprehensive view of spinal deformity.
- Integration of Clinical Knowledge: The methods, particularly the "Scapular Spine Triangle" and the fusion approach, explicitly incorporate established clinical observations and doctor expertise, enhancing interpretability and reliability.
- YOLOv8 for Keypoint Detection: The successful application of YOLOv8 for precise keypoint detection on back images is a foundational element for subsequent analyses.
- AMPD for Peak Detection: The use of AMPD for robust peak detection on back contours is crucial for accurate trunk rotation measurement, especially in the presence of noise.
Key Results
- Coronal Curvature Type Discrimination: Achieved an accuracy rate of 88.7%.
- Cobb Angle Measurement: Showed improved accuracy compared to existing deep learning and convolutional neural network methods, with a reported 17.0% improvement over one method and 7.6% over another.
- "Scapular Spine Triangle" Method: Demonstrated good performance, with accuracy improvements of 18.4% and 9.0% over comparable methods. However, it showed instability across different scoliosis types.
- Fusion Method: Achieved the highest accuracy (0.793) among the evaluated methods, outperforming others by a significant margin.
- Clinical Matched Data Verification: The proposed algorithm achieved an accuracy of 0.702 on a small set of clinically matched data.
- Trunk Rotation Measurement: The Prewitt operator for contour extraction resulted in the lowest average relative error (6.2%) for trunk rotation measurement, indicating high accuracy.
Limitations and Future Work
- Dataset Size and Diversity: While 3640 images were used, larger and more diverse datasets could further improve model generalization. The clinical matched data set was also small (25 patients).
- "Scapular Spine Triangle" Instability: The authors note that the "Scapular Spine Triangle" method's performance is unstable across different scoliosis types ("S" and multi-curve).
- Future Work: The authors envision deploying this multi-dimensional assessment algorithm into an application for easy patient use. They also plan to develop a remote monitoring and management system for continuous patient-doctor communication and guided rehabilitation, leveraging telemedicine.
Summary updated