Deep Learning Based Staging of Bone Lesions From Computed Tomography Scans

Samira Masoudi, Sherif Mehralivand, Stephanie A. Harmon, Nathan Lay, Liza Lindenberg, Esther Mena, Peter A. Pinto, Deborah E. Citrin, James L. Gulley, Bradford J. Wood, William L. Dahut, Ravi A. Madan, Ulas Bagci, Peter L. Choyke, Baris TurkbeyIEEE Access 2021 · DOI · OpenReview

TL;DR

This paper introduces a novel deep learning approach for classifying bone lesions in prostate cancer patients as benign or malignant using CT scans. By developing a comprehensive dataset and employing advanced deep learning architectures, the study achieves a high classification accuracy, paving the way for more precise and efficient cancer staging.

What Problem Does This Paper Address?

Prostate cancer is a leading cause of cancer death in men, and accurate staging is crucial for effective treatment. Metastases to the bone are common and can significantly impact a patient's quality of life. Distinguishing between benign and malignant bone lesions on CT scans is challenging due to the wide variety of benign lesion appearances that can mimic metastatic disease. Misclassification can lead to inappropriate treatment, underscoring the need for highly accurate diagnostic tools. Current methods often rely on manual interpretation, which can be time-consuming and prone to error. This paper aims to develop an automated, deep learning-based system to improve the accuracy and efficiency of bone lesion classification.

Methods

The researchers developed a deep learning framework for classifying bone lesions. Key aspects of their methodology include:

  • Lesion-Aware Data Stratification: To improve model generalizability, the dataset was split at the patient level, ensuring all lesions from a single patient were in the same split (training, validation, or test). This approach accounts for the fact that different lesion sub-categories might have unique features.
  • Patch and Lesion Extraction Strategies: They explored 12 different methods for extracting relevant information from CT scans, varying in size, shape, and whether they focused on 2D slices, 2.5D (slice + neighbors), or 3D volumes. This allowed them to investigate the impact of different feature representations.
  • Deep Learning Architectures: The study utilized state-of-the-art convolutional neural networks (CNNs), specifically ResNet-50, ResNeXt-50, and 3D ResNet-18. They experimented with both 2D/2.5D and 3D CNNs to capture different aspects of the lesion data.
  • Data Augmentation: Techniques like random horizontal flipping, zooming, rotation, and unfocused lesion extraction were employed to increase the robustness of the model and prevent overfitting.
  • Ensemble Model: The final proposed model is an ensemble of a 2D ResNet-50 (trained with texture and morphology features) and a 3D ResNet-18 (trained with volumetric information), combining their strengths for improved classification.

Datasets

The study utilized a newly curated dataset comprising 2,880 annotated bone lesions from CT scans of 114 prostate cancer patients. These annotations included full 3D segmentation and labels for benign or malignant. The dataset was further categorized into various benign and malignant sub-types. This dataset is presented as a valuable resource for research in this area.

Key Insights and Ideas

  • Texture is King (for 2D/2.5D): For 2D and 2.5D analysis, lesion texture emerged as the most significant factor for classification, outperforming morphology and location.
  • Volumetric Information Adds Value (for 3D): 3D analysis revealed that volumetric information, rather than just texture or morphology, plays a crucial role in classification.
  • 2D vs. 3D Perspectives: The study suggests that 2D and 3D deep learning models learn from fundamentally different feature sets, highlighting the benefit of combining them.
  • Lesion Location Can Be Detrimental: Counterintuitively, defining ROIs with explicit lesion location information sometimes degraded performance, possibly by diluting textural or morphological cues.
  • Ensemble Power: Combining the strengths of 2D and 3D models through an ensemble approach yielded the best overall performance, demonstrating the synergy between different feature extraction and modeling strategies.

Key Results

The proposed ensemble model achieved an accuracy of 92.2% for correctly classifying benign versus malignant bone lesions on the test set. This represents a significant achievement, particularly at the lesion level, and is claimed to be the highest reported accuracy for this task with such a comprehensive dataset. The study also found that texture and volumetric information were the most discriminative features, with morphology playing a more limited role in 2D/2.5D classification.

Limitations and Future Work

The authors acknowledge that the dataset, while comprehensive, is limited to 114 patients. They also note that 3D analysis, while offering a different perspective, experienced a drop in results compared to 2D analysis, potentially due to fewer training examples and the absence of lesion-based average voting. Future work includes cascading this AI classification method with a bone lesion detection algorithm to create a fully automated detection and classification system, aiming to further improve radiologist performance. They also suggest that incorporating regressed lesion coordinates directly into the classifier's dense layer could preserve local clues while maintaining texture and morphology information.

Summary updated

Abstract

In this study, we formulated an efficient deep learning-based classification strategy for characterizing metastatic bone lesions using computed tomography scans (CTs) of prostate cancer patients. For this purpose, 2,880 annotated bone lesions from CT scans of 114 patients diagnosed with prostate cancer were used for training, validation, and final evaluation. These annotations were in the form of lesion full segmentation, lesion type and labels of either benign or malignant. In this work, we present our approach in developing the state-of-the-art model to classify bone lesions as benign or malignant, where (1) we introduce a valuable dataset to address a clinically important problem, (2) we increase the reliability of our model by patient-level stratification of our dataset following lesion-aware distribution at each of the training, validation, and test splits, (3) we explore the impact of lesion texture, morphology, size, location, and volumetric information on the classification performance, (4) we investigate the functionality of lesion classification using different algorithms including lesion-based average 2D ResNet-50, lesion-based average 2D ResNeXt-50, 3D ResNet-18, 3D ResNet-50, as well as the ensemble of 2D ResNet-50 and 3D ResNet-18. For this purpose, we employed a train/validation/test split equal to 75%/12%/13% with several data augmentation methods applied to the training dataset to avoid overfitting and to increase reliability. We achieved an accuracy of 92.2% for correct classification of benign vs. malignant bone lesions in the test set using an ensemble of lesion-based average 2D ResNet-50 and 3D ResNet-18 with texture, volumetric information, and morphology having the greatest discriminative power respectively. To the best of our knowledge, this is the highest ever achieved lesion-level accuracy having a very comprehensive data set for such a clinically important problem. This level of classification performance in the early stages of metastasis development bodes well for clinical translation of this strategy.
BibTeX
@article{masoudi2021deep,
  author = {Samira Masoudi and Sherif Mehralivand and Stephanie A. Harmon and Nathan Lay and Liza Lindenberg and Esther Mena and Peter A. Pinto and Deborah E. Citrin and James L. Gulley and Bradford J. Wood and William L. Dahut and Ravi A. Madan and Ulas Bagci and Peter L. Choyke and Baris Turkbey},
  title = {{Deep Learning Based Staging of Bone Lesions From Computed Tomography Scans}},
  journal = {IEEE Access},
  year = {2021},
  doi = {10.1109/access.2021.3074051},
}
Figure