TL;DR
This paper introduces DisAsymNet, a novel framework that disentangles asymmetrical abnormalities from normal tissue in bilateral mammograms. By leveraging self-adversarial learning and a synthesis approach, DisAsymNet achieves state-of-the-art performance in abnormality classification, segmentation, and localization, offering more interpretable diagnostic cues.
What Problem Does This Paper Address?
Mammography plays a crucial role in breast cancer screening, and identifying asymmetries between the left and right breasts is a key diagnostic indicator for radiologists. However, current AI models often struggle to effectively leverage this bilateral asymmetry, focusing more on general multi-view analysis rather than direct comparison. A significant challenge is understanding what a mammogram would look like if the asymmetrical abnormalities were removed, which could provide valuable insights for diagnosis and model interpretability. Existing methods lack the ability to reconstruct a truly symmetric breast image, and this paper aims to fill that gap by disentangling abnormalities from normal tissue.
Methods
The core of DisAsymNet is a two-module framework:
- Asymmetric Transformer-based Classification (AsyC): This module mimics how radiologists compare bilateral mammograms. It uses shared encoders to extract features from both left and right views and then employs an "asymmetric transformer" that combines self-attention (SA) and cross-attention (CA). This allows the model to aggregate information from both self and contralateral views, enhancing the side-by-side comparison. An online Class Activation Mapping (CAM) module is used to generate heatmaps for segmentation and localization.
- Asymmetric Abnormality Disentanglement (AsyD): This module focuses on separating abnormal regions from normal tissue. It uses U-Net-like decoders to generate both abnormal and normal images for each side. To address the lack of pixel-level annotations for asymmetry, the authors introduce a "Synthesis" method. This involves randomly inserting synthetic tumors into normal mammograms to create supervised training data for reconstruction. Additionally, a self-adversarial learning scheme is employed, where a frozen discriminator guides the disentanglement process, and feedback from the classification module refines the abnormality detection.
Datasets
The study utilizes four mammography datasets:
- INBreast: Contains 115 exams with BI-RADS labels and pixel-wise annotations.
- DDSM: Includes 2,620 cases with annotations generated by a mammographer.
- VinDr-Mammo: Features 5,000 cases with BI-RADS assessments and bounding box annotations.
- In-house dataset: A large dataset of 43,258 exams, used primarily for classification tasks due to a lack of detailed annotations.
Each dataset was split into training, validation, and testing sets.
Key Insights and Ideas
- Mimicking Radiologist's Workflow: The AsyC module's design, incorporating both self-attention and cross-attention, directly reflects how radiologists analyze mammograms by considering both unilateral features and bilateral comparisons.
- The Power of "What If": The central question of "what would a symmetric mammogram look like?" is addressed by the AsyD module, which aims to reconstruct a normal breast view. This is crucial for interpretability and understanding the model's decision-making process.
- Self-Adversarial Learning for Weak Supervision: The use of self-adversarial learning, particularly with a frozen discriminator, is a clever way to impose constraints and guide the generator without requiring extensive pixel-level ground truth for asymmetry.
- Synthesis for Reconstruction Supervision: The "Synthesis" method provides a practical and cost-effective way to generate supervised signals for the disentanglement task, overcoming the scarcity of real-world pixel-level asymmetry annotations.
Key Results
- Classification: DisAsymNet consistently outperforms existing single-view and multi-view methods in abnormal classification tasks across all four datasets, achieving high AUC scores. For example, on the INBreast dataset, DisAsymNet achieves an AUC of 0.951 for abnormal classification, significantly higher than other methods.
- Segmentation and Localization: The framework demonstrates superior performance in weakly supervised segmentation and localization tasks compared to other methods, achieving an IoU of 0.461 and Dice score of 0.601 on the INBreast dataset.
- Ablation Studies: Ablation studies confirm the effectiveness of each component: "AsyC" alone surpasses some existing methods, and the addition of "AsyD" and "Synthesis" further boosts performance.
- Robustness: The model shows robustness across different dataset sizes, maintaining its performance advantage even with reduced training data.
- Qualitative Results: Visualizations show that DisAsymNet effectively removes asymmetrical abnormalities while preserving normal tissue, producing more natural-looking symmetric mammograms.
Limitations and Future Work
- Synthetic Artifacts: While the "Synthesis" method is effective, the paper notes that without pixel-level ground truth, the generator can sometimes over-remove abnormalities, leading to "black holes" or unnaturally dark areas. The incorporation of synthetic asymmetrical Bi-MG during training helps mitigate this, leading to more natural symmetric tissue generation.
- Future Directions: The authors suggest that the potential of leveraging asymmetry could be further investigated in other clinical tasks, such as breast cancer risk prediction. They also highlight the interpretability benefits of generating normal symmetric breast views.
Summary updated