Abstract
Background: Significant differences exist in classification outcomes for radiologists using ultrasonography-based breast imaging-reporting and data systems for diagnosing category 3–5 (BI-RADS-US 3–5) breast nodules, due to a lack of clear and distinguishing image features. As such, this study investigates the use of a transformer-based computer-aided diagnosis (CAD) model for improved BI-RADS-US 3–5 classification consistency.
Methods: Five radiologists independently performed BI-RADS-US annotations on a breast ultrasonography image set collected from 20 hospitals in China. The data were divided into training, validation, testing, and sampling sets. The trained transformer-based CAD model was then used to classify test images, for which sensitivity, specificity, and accuracy were calculated. Variations in these metrics among the 5 radiologists were analyzed by referencing BI-RADS-US classification results for the sampling test set, provided by CAD, to determine whether classification consistency (the kappa value), sensitivity, specificity, and accuracy had improved.
Results: Classification accuracy for the CAD model applied to the test set was 95.7% for category 3 nodules, 97.6% for category 4A nodules, 95.60% for category 4B nodules, 94.2% for category 4C nodules, and 97.5% for category 5 nodules. Adjustments were made to 1,583 nodules, as 905 were classified to a higher category and 678 to a lower category in the sampling test set. As a result, the accuracy, sensitivity, and specificity of classification by each radiologist improved, with the consistency (kappa values) for all radiologists increasing to >0.60.
Conclusions: The proposed transformer-based CAD model improved BI-RADS-US 3–5 nodule classification by individual radiologists and increased diagnostic consistency.
Introduction
In breast cancer screening, the Breast Imaging Reporting and Data System (BI-RADS) for ultrasound is the primary tool for standardizing reports. However, categories 3, 4, and 5—which range from "probably benign" to "highly suggestive of malignancy"—suffer from significant inter-observer variability. Category 4 is particularly problematic, covering a massive malignancy risk range (2% to 95%) and subdivided into 4a, 4b, and 4c. Because these subcategories lack objective mathematical definitions, radiologists often disagree on the same image, leading to unnecessary biopsies or delayed treatments. This study introduces a Transformer-based computer-aided diagnosis (CAD) model designed to act as an objective "referee" to stabilize these classifications.
Methods: The Power of Transformers
The researchers utilized a Hierarchical Vision Transformer architecture, moving away from traditional Convolutional Neural Networks (CNNs). While CNNs excel at detecting local textures, Transformers use a "self-attention" mechanism that captures global shapes and long-range dependencies. In medical imaging, the overall morphology of a nodule is often more diagnostic than local pixel patterns.
The model processes images at multiple scales to understand both fine details and broader anatomical context. To test its clinical utility, five radiologists with varying experience (8 to 19 years) performed a "human-in-the-loop" evaluation, re-classifying nodules after viewing the CAD model’s suggestions to measure the "AI-reference effect."
Datasets and Pathological Ground Truth
The study is notable for its scale and diversity, utilizing 21,332 ultrasound images of 5,057 breast nodules from 3,978 patients. Data was collected from 20 different hospitals across China using 34 different types of ultrasound machines, ensuring the model's robustness. Crucially, every nodule in the dataset was surgically removed and confirmed via pathological examination, providing a definitive "gold standard."
As shown in the table above, the dataset included a wide variety of pathological results, with non-specific invasive carcinoma and fibroadenomas being the most common malignant and benign findings, respectively.
Key Results and Radiologist Improvement
The CAD model demonstrated high accuracy across all critical BI-RADS categories:
- 95.7% for Category 3.
- 97.6% for Category 4a.
- 97.5% for Category 5.
Beyond raw accuracy, the AI acted as a "consistency engine." Before referencing the AI, radiologist agreement (Kappa value) was as low as 0.33 (poor/moderate). After referencing the CAD results, agreement scores improved to a range of 0.67–0.85 (strong consistency).
The AI also helped correct the "gray area" of diagnosis. Radiologists adjusted 1,583 classifications after seeing the AI's output. Specifically, 272 nodules were downgraded from 4a to 3; importantly, 81.99% of those downgraded nodules were pathologically confirmed to be benign, suggesting the AI can successfully help avoid unnecessary invasive procedures.
Discussion and Conclusion
The Transformer-based CAD model effectively addresses the subjectivity inherent in BI-RADS-US 3–5 classification. By focusing on global shape recognition rather than just ultrasound texture "noise," the model provides a reliable reference that narrows the subjectivity gap among radiologists.
While the study is limited by its retrospective nature and use of static 2D images, it highlights the potential for AI to standardize breast cancer screening. Future work will look toward multi-modal AI—combining 2D images with blood flow (Color Doppler) and tissue stiffness (Elastography) data—to further refine diagnostic precision.
Summary updated