TL;DR
This paper introduces a deep learning (DL) model designed to triage screening mammograms, identifying a subset as likely cancer-free. This approach significantly reduces radiologist workload and improves specificity without compromising sensitivity, offering a promising avenue for more efficient breast cancer screening.
What Problem Does This Paper Address?
Mammography is a critical tool for early breast cancer detection, but its effectiveness is hampered by variations in radiologist performance and workflow inefficiencies. Specifically, radiologists face a high volume of screening mammograms, leading to potential fatigue and errors. While existing computer-aided detection (CAD) systems aim to improve sensitivity, they often don't address radiologist specificity or overall workflow efficiency. Double reading, another method to improve accuracy, further burdens the workflow. This paper tackles the challenge of improving both the efficiency and accuracy of mammogram interpretation by developing a DL model that can confidently identify "cancer-free" cases, allowing radiologists to focus their expertise on more complex or suspicious scans.
Methods
The researchers developed a deep learning model based on a convolutional neural network (ResNet18) trained to predict the likelihood of a breast developing cancer within one year. The model was trained on a large dataset of screening mammograms and their corresponding cancer outcomes. For the triage application, a high-sensitivity threshold was established on the validation set. This threshold was set to the minimum probability score that corresponded to a radiologist's true-positive assessment. In a simulated workflow, mammograms falling below this threshold were triaged as "cancer-free" and skipped by radiologists. Mammograms above the threshold were then read by radiologists as usual. The study then compared the performance metrics (sensitivity, specificity, and percentage of mammograms read) of this simulated DL-triage workflow against the standard workflow where radiologists read all mammograms.
Datasets
The study utilized a retrospective dataset of 223,109 consecutive screening mammograms from 66,661 women, collected between January 2009 and December 2016. This dataset was split into training (212,272 mammograms from 56,831 patients), validation (25,999 mammograms from 7,021 patients), and testing (26,540 mammograms from 7,176 patients) sets. Cancer outcomes were obtained through linkage to regional tumor registries and pathology reports. The dataset was carefully curated to include women with either a diagnosis of breast cancer within one year or at least one year of imaging follow-up.
Key Insights and Ideas
- Shifting Focus from Detection to Triage: Instead of solely aiming to "detect" suspicious findings like traditional CAD, this DL model focuses on "triaging" – confidently identifying likely negative cases to reduce workload.
- Leveraging "Cancer-Free" Confidence: The model's strength lies in its ability to achieve high confidence in identifying mammograms that are very unlikely to harbor cancer, enabling a significant portion of the workload to be bypassed by human readers.
- Complementary to CAD: This triage approach is presented as complementary to CAD systems, potentially working together to optimize the screening process.
- Broad Applicability: The model demonstrated consistent performance across various demographic groups (age, race) and breast densities, suggesting its potential for widespread use.
Key Results
In the simulated DL-triage workflow:
- The model triaged approximately 19.3% of mammograms as cancer-free, reducing the number of mammograms read by radiologists.
- Specificity improved significantly from 93.5% to 94.2% (P = .002).
- Sensitivity remained non-inferior, with a slight decrease from 90.6% to 90.1% (P < .001), well within the 5% margin.
- The workload reduction was substantial, with radiologists reading only 80.7% of mammograms.
- The model showed similar predictive accuracy across different age groups and breast densities, with AUCs generally ranging from 0.71 to 0.86.
Limitations and Future Work
The authors acknowledge several limitations:
- Retrospective Simulation: The study relied on a retrospective simulation, assuming radiologists would interpret remaining images the same way. A prospective trial is needed to confirm these findings in real-world clinical practice.
- Single Institution and Vendor: The model was developed and tested at a single academic institution using mammograms from a single vendor (Hologic). External validation with diverse populations and mammography equipment is crucial.
- Potential for Radiologist Bias: The knowledge of risk assessment and reduced workload might influence radiologist performance in ways not captured by this simulation.
Future work should focus on prospective validation, exploring different triage thresholds, and investigating the model's performance with mammograms from diverse vendors. The authors also suggest that this model could be adapted for scenarios with limited access to specialized radiologists or for identifying cases suitable for single-human assessment in regions where double reading is common.
Summary updated