Dual Convolutional Neural Networks for Breast Mass Segmentation and Diagnosis in Mammography

Heyi Li, Dongdong Chen, William H. Nailon, Mike E. Davies, David I. Laurenson

TL;DR

This paper introduces DUALCORENET, a novel end-to-end deep learning framework that simultaneously segments breast masses and diagnoses them as benign or malignant in mammograms. By employing a dual-path architecture, it effectively combines textural and shape features, outperforming existing methods in both segmentation and classification tasks.

What Problem Does This Paper Address?

Breast cancer screening through mammography is crucial, but manual interpretation is tedious and prone to errors. Computer-aided diagnosis (CAD) systems aim to assist radiologists, but traditional methods often rely on hand-engineered features and struggle to integrate segmentation and diagnosis effectively. Existing deep learning approaches often treat these as separate stages or require extensive pixel-level annotations, which are difficult to obtain. This paper addresses the challenge of developing a unified, end-to-end deep learning system that can accurately segment breast masses and simultaneously predict their malignancy, leveraging both visual and structural information.

Methods

DUALCORENET employs a novel dual-path architecture to tackle the segmentation and classification problems concurrently.

  • Locality Preserving Learner (LPL): This path focuses on extracting hierarchical and intrinsic features from larger-scale Regions of Interest (ROIs) that include surrounding tissues. It utilizes a deep convolutional neural network (CNN) backbone inspired by architectures like VGG and ResNet, incorporating residual connections to improve training and generalization. The LPL path is designed to capture textural and contextual information crucial for classification.
  • Conditional Graph Learner (CGL): This path is dedicated to generating geometrical features by modeling pixel-wise image-to-mask correlations. It addresses the challenge of precise segmentation, especially for irregular shapes and fine boundaries, by integrating a Conditional Random Field (CRF) inference layer within a CNN. This approach aims to preserve structural information and produce sharp segmentations.
  • Feature Fusion: The features extracted from both the LPL and CGL paths are fused in a dedicated module. This fusion allows the model to leverage both the semantic (textural) and geometric (shape) information, leading to improved diagnostic performance.
  • End-to-End Training: The entire DUALCORENET is trained end-to-end, optimizing a combined loss function that includes contributions from both the LPL and CGL paths, as well as a fusion loss for classification.

Datasets

The DUALCORENET was evaluated on two widely used public mammography datasets:

  • CBIS-DDSM: A modernized subset of the Digital Database for Screening Mammography, containing digitized mammograms. The paper uses the provided training and test set division.
  • INbreast: A Full Field Digital Mammography (FFDM) dataset acquired at a Breast Centre in Portugal, featuring pixel-level annotations by human experts. The dataset was split into training and testing sets with an 80%:20% ratio.

Key Insights and Ideas

  • Dual-Problem Decoupling: The core idea of treating segmentation and classification as intertwined "dual problems" within a single network is a key innovation. This allows for a more holistic understanding of the mass.
  • Complementary Feature Learning: The LPL path excels at capturing broader contextual and textural information for diagnosis, while the CGL path focuses on precise boundary and shape details for segmentation. Their integration is crucial for superior performance.
  • CRF Integration for Segmentation: Using CRF as an integral part of the CNN architecture (rather than just a post-processing step) helps in achieving more balanced and accurate segmentations, especially for complex lesions.
  • Multi-Scale ROIs: The use of multi-scale ROIs as input to different paths acknowledges that different scales of information are relevant for segmentation and classification.
  • Residual Learning Benefits: The paper demonstrates that residual connections significantly improve the generalization ability of the DUALCORENET, leading to more stable validation loss.

Key Results

DUALCORENET achieved state-of-the-art or highly competitive results on both datasets:

  • Segmentation:
    • Achieved the best mammography segmentation performance on the INbreast dataset with a Dice coefficient of 93.69%.
  • On the DDSM dataset, it achieved a Dice coefficient of 92.17%, outperforming previous methods.
  • Classification:
    • On the INbreast dataset, it achieved an AUC score of 0.93.
  • On the DDSM dataset, it achieved an AUC score of 0.85.
  • The paper highlights that the DUALCORENET is the only algorithm that achieved leading performance in both segmentation and classification simultaneously.

Limitations and Future Work

  • Dataset Quality: The paper notes that DUALCORENET performs better on higher-quality datasets like INbreast, suggesting that data quality plays a significant role.
  • Training Loss Ablation: While the paper explores different loss configurations, it mentions that using only the fusion loss led to a non-converged model, underscoring the necessity of the LPL and CGL path losses.
  • Future Work: The authors suggest that further exploration of different feature fusion strategies and potentially incorporating more advanced graphical models could lead to even greater improvements. The current work focuses on a dual-path approach, and future research could explore multi-path architectures for even more complex scenarios.

Summary updated

Abstract

Deep convolutional neural networks (CNNs) have emerged as a new paradigm for Mammogram diagnosis. Contemporary CNN-based computer-aided-diagnosis systems (CADs) for breast cancer directly extract latent features from input mammogram image and ignore the importance of morphological features. In this paper, we introduce a novel end-to-end deep learning framework for mammogram image processing, which computes mass segmentation and simultaneously predicts diagnosis results. Specifically, our method is constructed in a dual-path architecture that solves the mapping in a dual-problem manner, with an additional consideration of important shape and boundary knowledge. One path, called the Locality Preserving Learner (LPL), is devoted to hierarchically extracting and exploiting intrinsic features of the input. Whereas the other path, called the Conditional Graph Learner (CGL), focuses on generating geometrical features via modeling pixel-wise image to mask correlations. By integrating the two learners, both the cancer semantics and cancer representations are well learned, and the component learning paths in return complement each other, contributing an improvement to the mass segmentation and cancer classification problem at the same time. In addition, by integrating an automatic detection set-up, the DualCoreNet achieves fully automatic breast cancer diagnosis practically. Experimental results show that in benchmark DDSM dataset, DualCoreNet has outperformed other related works in both segmentation and classification tasks, achieving 92.27% DI coefficient and 0.85 AUC score. In another benchmark INbreast dataset, DualCoreNet achieves the best mammography segmentation (93.69% DI coefficient) and competitive classification performance (0.93 AUC score).
BibTeX
@article{li2021dual,
  author = {Heyi Li and Dongdong Chen and William H. Nailon and Mike E. Davies and David I. Laurenson},
  title = {{Dual Convolutional Neural Networks for Breast Mass Segmentation and Diagnosis in Mammography}},
  journal = {IEEE Transactions on Medical Imaging},
  year = {2021},
  doi = {10.1109/tmi.2021.3102622},
}
Figure