A generative adversarial network-based abnormality detection using only normal images for model training with application to digital breast tomosynthesis

Albert Swiecicki, Nicholas Konz, Mateusz Buda, Maciej A. MazurowskiScientific 2021 · DOI

TL;DR

This paper introduces a novel approach for detecting abnormalities in medical images, specifically digital breast tomosynthesis (DBT), by training a generative adversarial network (GAN) to "inpaint" missing image regions using only normal images. The core idea is that the GAN will struggle to accurately reconstruct abnormal regions, leading to a measurable error that can flag potential issues. This method is crucial for rare disease detection where abnormal samples are scarce.

What Problem Does This Paper Address?

Detecting abnormalities in medical imaging, such as cancer in mammograms or DBT scans, is a critical task. However, a significant challenge in training deep learning models for this purpose is the extreme class imbalance: the vast majority of screening images are normal, while only a tiny fraction contain abnormalities (less than 1% for DBT). This scarcity of positive examples makes it difficult for models to learn what constitutes an abnormality. Traditional supervised methods require many labeled abnormal cases, which are hard to obtain. This paper tackles this problem by proposing a way to leverage the abundance of normal data to detect the rare abnormal cases, without needing any abnormal images during the primary training phase.

Methods

The authors propose using a generative adversarial network (GAN) for image completion (inpainting). The core idea is to train a GAN on a large dataset of normal DBT images. During training, parts of these normal images are masked out, and the GAN learns to reconstruct the missing regions based on the surrounding healthy tissue.

The specific architecture involves a two-phase generator with dilated convolutional blocks for a coarse prediction and a refinement network with contextual attention for finer details. This generator is paired with both local and global discriminators to ensure consistency between the completed region and the overall image.

For abnormality detection, the trained GAN is used in an inference setting. A sliding window approach is employed: parts of a test image are masked and inpainted by the GAN. The "error" in the reconstruction is then measured. The authors propose using a combination of Mean Squared Error (MSE) and discriminator loss, termed DMSE, as the primary metric. A high reconstruction error in a specific region is hypothesized to indicate an abnormality.

Datasets

The study utilizes a dataset of digital breast tomosynthesis (DBT) screening studies collected at Duke University.

  • Training and Validation: Consisted of 18,232 reconstruction volumes from normal cases.
  • Testing: Utilized 70 volumes from cancer cases. The dataset contains 19,230 reconstructed volumes from 4,348 patients in total. Cancerous masses and architectural distortions were marked with bounding boxes by radiologists. The data is made publicly available on The Cancer Imaging Archive.

Key Insights and Ideas

  • Leveraging Normal Data for Anomaly Detection: The central insight is that a GAN trained exclusively on normal data will inherently struggle to reconstruct abnormal regions. This "failure to reconstruct" becomes a signal for abnormality.
  • Inpainting as an Anomaly Detector: The paper reframes image completion not just as a generative task but as a powerful tool for anomaly detection, especially in data-scarce scenarios.
  • DMSE as a Robust Error Metric: The combined MSE and discriminator loss (DMSE) proved to be more effective than individual metrics for flagging abnormalities, suggesting that a combination of pixel-level reconstruction fidelity and contextual realism is key.
  • Contextual Attention for Realistic Inpainting: The use of contextual attention in the refinement network helps the GAN generate more realistic completions, which is crucial for accurately identifying deviations from normal patterns.

Key Results

  • The proposed method demonstrated that locations containing cancer were associated with a notably higher completion error than non-cancer locations.
  • The mean error ratio (DMSE inside bounding boxes vs. outside) for cancerous regions was 2.77.
  • The DMSE metric outperformed individual MSE and discriminator loss metrics in detecting abnormalities.
  • The best performance was achieved with a field of view of 256x256 pixels and a mask size of 128x128 pixels.
  • The model was able to generate realistic completions of normal breast tissue structures like veins, while struggling with abnormal regions.

Limitations and Future Work

  • Limited Test Set Size: The number of positive cases in the test set was small (70 volumes), which, while sufficient for an overview, limits the statistical power of the findings.
  • Annotation Scope: The dataset lacked annotations for all types of abnormalities (e.g., postoperative clips), preventing a comprehensive evaluation of detection for all potential anomalies.
  • Computational Cost: Generating heatmaps required thousands of model runs per image, indicating a significant computational burden.
  • No-Padding Approach: The method omits boundary parts of the image during detection due to the no-padding approach, which could miss abnormalities near the edges.
  • Future Work: The authors suggest oversampling unusual benign structures during training to address false positives. Combining this approach with fully supervised methods could further improve performance, especially for scarce training signals. Further research is needed to define classification metrics (e.g., accuracy, ROC curves) by establishing appropriate error thresholds for abnormality detection.

Summary updated

Abstract

Deep learning has shown tremendous potential in the task of object detection in images. However, a common challenge with this task is when only a limited number of images containing the object of interest are available. This is a particular issue in cancer screening, such as digital breast tomosynthesis (DBT), where less than 1% of cases contain cancer. In this study, we propose a method to train an inpainting generative adversarial network to be used for cancer detection using only images that do not contain cancer. During inference, we removed a part of the image and used the network to complete the removed part. A significant error in completing an image part was considered an indication that such location is unexpected and thus abnormal. A large dataset of DBT images used in this study was collected at Duke University. It consisted of 19,230 reconstructed volumes from 4348 patients. Cancerous masses and architectural distortions were marked with bounding boxes by radiologists. Our experiments showed that the locations containing cancer were associated with a notably higher completion error than the non-cancer locations (mean error ratio of 2.77). All data used in this study has been made publicly available by the authors.
BibTeX
@inproceedings{swiecicki2021generative,
  author = {Albert Swiecicki and Nicholas Konz and Mateusz Buda and Maciej A. Mazurowski},
  title = {{A generative adversarial network-based abnormality detection using only normal images for model training with application to digital breast tomosynthesis}},
  booktitle = {Scientific},
  year = {2021},
  doi = {10.1038/s41598-021-89626-1},
}
Figure