TL;DR
This paper introduces a novel "mask branch network" that leverages information from radiology reports to create template masks for breast masses. This weakly supervised approach significantly improves the performance of existing deep learning models for classifying masses in 3D Automated Breast Ultrasound (ABUS) images, without requiring manual segmentation.
What Problem Does This Paper Address?
Automated Breast Ultrasound (ABUS) is a valuable tool for breast cancer screening, but accurately detecting and classifying masses can be challenging. Masses often appear as irregular, hypoechoic areas that are difficult to distinguish from background noise and shadowing. While deep learning models, particularly Convolutional Neural Networks (CNNs), have shown promise, they often struggle with the subtle features and 3D context of these masses, leading to false positives. A key challenge is obtaining accurate segmentation masks for training these models, which is a time-consuming and expert-intensive task. This paper aims to overcome this by using readily available information from radiology reports to guide the learning process.
Methods
The core innovation is a "mask branch network" (MBN) that is integrated into a standard CNN classifier (the "main network"). The MBN operates in parallel with the main network during training. It takes intermediate features from the main network and generates a voxel-level probability map, which acts as a surrogate for a segmentation mask.
Crucially, instead of relying on manually annotated masks, the MBN uses "template masks" derived from radiology reports. These reports typically contain information about the lesion's location, diameter, and orientation. The authors developed a rule-based method to construct these template masks, treating the mass as a sphere or ellipsoid based on its orientation relative to the breast skin.
The MBN's output is compared to this template mask using a pixelwise cross-entropy loss (L_mask). This mask loss is then combined with the main classification loss (L_main) using an uncertainty weighting method. This allows the model to adaptively learn the relative importance of the classification and segmentation tasks, leading to improved feature extraction focused on the mass region. During inference, only the main network is used, making the approach efficient.
Datasets
The authors built their own dataset from a tertiary hospital, comprising ABUS images from 363 patients. This dataset includes 434 mass lesions and 3,907 non-mass lesions (cropped from normal scans). The center coordinates and diameters of masses were obtained from radiology reports. The data was split into training, validation, and test sets.
Key Insights and Ideas
- Weak Supervision is Powerful: By leveraging existing radiology report information, the authors demonstrate that detailed manual segmentation is not always necessary for effective deep learning. This significantly reduces the annotation burden.
- Template Masks Guide Attention: The template masks, derived from reports, effectively guide the CNN to focus its attention on the relevant mass regions, leading to more discriminative features.
- Uncertainty Weighting Balances Tasks: Combining classification and "segmentation-like" losses with uncertainty weighting allows the model to learn a more robust representation by balancing the contributions of each task.
- Template Mask Features Matter: The study shows that incorporating location, size, and orientation from radiology reports into the template mask progressively improves performance, with the combination of all three features yielding the best results.
Key Results
The proposed mask branch network, when integrated with existing classifiers like DenseNet and ResNet, significantly improved classification performance. For example, when using DenseNet as the backbone, the addition of the MBN increased sensitivity from 56.3% to 75.0%, specificity from 86.3% to 92.5%, and AUC from 0.855 to 0.9252. Similar improvements were observed with the ResNet backbone. The study also showed that using location, size, and orientation features for template mask creation led to better results than using location alone or no mask at all.
Limitations and Future Work
The authors acknowledge that the template mask is a simplified representation and may not perfectly capture the true boundaries of complex masses. They also note that the performance of the template mask itself depends on the accuracy and completeness of the information in the radiology reports.
For future work, the authors plan to explore variations of the branch network architecture to further optimize performance. They also intend to apply their method to more challenging ABUS problems, such as classifying malignancy.
Summary updated