Computational mammography using deep neural networks

Anastasia Dubrovina, Pavel Kisilev, Boris Ginsburg, Sharbell Y. Hashoul, Ron KimmelComput. Methods Biomech. Biomed. Eng. Imaging Vis 2018 · DOI

TL;DR

This paper introduces a novel deep learning framework for automatically classifying breast tissue in mammograms, aiming to improve early breast cancer detection. By leveraging Convolutional Neural Networks (CNNs) and a patch-wise training approach, they achieve efficient and accurate segmentation of different tissue types, significantly speeding up the classification process.

What Problem Does This Paper Address?

Breast cancer remains a leading cause of death among women, making early detection crucial. Mammography is a primary tool for this, but manual analysis by radiologists is time-consuming and can be subjective. Automatic classification and segmentation of breast tissues within mammograms could significantly aid radiologists by highlighting suspicious areas and reducing their workload. The challenge lies in accurately distinguishing between various tissue types (pectoral muscle, fibroglandular tissue, nipple, and general breast tissue) from complex medical images, especially with limited annotated data.

Methods

The authors propose a supervised deep learning framework using Convolutional Neural Networks (CNNs). Key aspects of their approach include:

  • Patch-wise Training: To overcome the challenge of limited annotated data, the CNN is trained on overlapping image patches. This effectively creates a much larger training dataset.
  • Convolutional Layers for Classification: Instead of traditional fully connected layers at the end of the network, they use convolutional layers. This allows the network to output dense predictions for the entire image, enabling a much faster pixel-wise classification.
  • Fully Convolutional Network (FCN) Conversion: To further accelerate the classification of whole images, they convert the fully connected layers into convolutional layers, creating a fully convolutional network. This allows for efficient processing of images of arbitrary sizes.
  • Shift-and-Stitch Method: To obtain dense predictions for the entire image from the FCN, they employ a shift-and-stitch technique, where shifted versions of the input image are processed and their outputs are interlaced.
  • Post-processing: A post-processing step is applied to refine the raw DNN output, incorporating physiological breast structure knowledge to correct fragmented regions and ensure anatomically plausible segmentation.

Datasets

The study used a dataset of 40 manually segmented mediolateral oblique (MLO) view mammograms. These images were aligned to ensure spatial consistency. For training, approximately 800,000 patches were extracted, with an equal number of patches centered on pixels belonging to each of the four tissue classes. A leave-one-subject-out cross-validation procedure was used for evaluation.

Key Insights and Ideas

  • Patch-wise training is a viable strategy for limited medical datasets: By breaking down images into patches, the authors effectively created a large enough dataset to train a deep CNN without requiring massive amounts of fully annotated images.
  • Replacing fully connected layers with convolutional layers dramatically speeds up inference: This is a crucial insight for real-time or near-real-time medical image analysis, reducing classification time from minutes to seconds.
  • Incorporating anatomical priors in post-processing is essential: While the CNN can learn features, the raw output can be fragmented. Post-processing based on known breast anatomy significantly improves the quality and interpretability of the segmentation.
  • Spatial information is tricky to integrate effectively: Despite attempts to incorporate pixel coordinates, the authors found it did not improve accuracy, suggesting that current methods for integrating spatial context in this patch-based approach are not optimal.

Key Results

  • Speedup: The fully convolutional network achieved a classification time of approximately 1.8 seconds per image, a significant improvement over the per-patch neural network approach which took 114 seconds. This represents nearly a two-order-of-magnitude speedup.
  • Accuracy (Dice Coefficients):
    • Pectoral muscle: 0.85
    • Fibroglandular tissue: 0.61
    • Nipple: 0.56
    • Average over all tissues: 0.71
  • Comparison: The proposed method achieved a higher Dice coefficient for pectoral muscle segmentation (0.85) compared to a related work [6] that reported 0.54 without atlas information.
  • Qualitative Results: The segmentation results show good detection of the pectoral muscle, with reasonable, though less accurate, detection of fibroglandular tissue and nipples.

Limitations and Future Work

  • Spatial Information Integration: The authors acknowledge that incorporating spatial pixel location information did not improve segmentation accuracy and requires further research to devise a correct method.
  • Appearance Variability: The lower accuracy for fibroglandular tissue and nipples is attributed to their high appearance variability across different regions.
  • Future Directions:
    • Integrating spatial smoothness into the classification process.
  • Extending the framework to explore recurrent neural networks (RNNs) instead of fully convolutional setups.
  • Using larger annotated datasets.
  • Incorporating breast structure priors and smart data normalization.

Summary updated

BibTeX
@article{dubrovina2018computational,
  author = {Anastasia Dubrovina and Pavel Kisilev and Boris Ginsburg and Sharbell Y. Hashoul and Ron Kimmel},
  title = {{Computational mammography using deep neural networks}},
  journal = {Comput. methods Biomech. Biomed. Eng. Imaging Vis.},
  year = {2018},
  doi = {10.1080/21681163.2015.1131197},
}
Figure