Decoupling Inherent Risk and Early Cancer Signs in Image-Based Breast Cancer Risk Models

Yue Liu, Hossein Azizpour, Fredrik Strand, Kevin Smith

This paper, titled "Decoupling Inherent Risk and Early Cancer Signs in Image-based Breast Cancer Risk Models," presents a critical warning for the field of medical AI: more data isn't always better if that data confuses two distinct biological signals.

TL;DR

Current AI models for breast cancer risk often "conflate" two different things: the long-term predisposition to develop cancer (inherent risk) and the presence of a tumor that hasn't been officially diagnosed yet (early signs). By explicitly decoupling these signals during training, researchers found they could build more accurate models for both short-term detection and long-term risk assessment.

What Problem Does This Paper Address?

In the world of mammography, there is a subtle but vital distinction between risk prediction and cancer detection.

  • Risk prediction asks: "What is the likelihood this woman will develop cancer in the next 5 to 10 years?"
  • Cancer detection asks: "Is there a tumor in this image right now?"

The authors argue that many "state-of-the-art" deep learning models are trained "carelessly" by using all available images from patients who eventually develop cancer. Because a tumor is a much "louder" signal for a neural network to find than the subtle textures associated with long-term risk, the models become hyper-focused on finding existing tumors. This makes them appear highly accurate in the short term but surprisingly poor at predicting actual long-term risk. Worse, it could lead a doctor to tell a patient they have "high long-term risk" when they actually have a tumor that needs immediate treatment.

Methods

The researchers trained three different versions of a ResNet50 neural network to compare how data selection affects what the AI learns:

  1. Inherent Risk Model: Trained only on "clean" images—either from the breast opposite to the one that developed cancer (contralateral) or images taken more than a year before diagnosis.
  2. Cancer Signs Model: Trained specifically on images taken within one year of a cancer diagnosis, where early signs are likely present.
  3. Conflated Model: The "standard" approach, trained on all images from cancer patients regardless of when they were taken.

To understand what the models were looking at, the team used Grad-CAM (Gradient-weighted Class Activation Mapping) to visualize which parts of the mammogram triggered the AI's "high risk" score.

Datasets

The study utilized the CSAW dataset, a massive population-based screening cohort from Sweden.

  • Total Images: The final training set included 138,032 mammograms from over 15,000 women.
  • Test Set: 6,436 mammograms from 731 women.
  • Baseline: The models were compared against a standard mammographic density baseline (using LIBRA software), which is the current clinical gold standard for risk.

Key Insights and Ideas

  • The "Easy Signal" Trap: Neural networks are "lazy" in a mathematical sense—they will converge on the easiest features to identify. Detecting a tumor is easier than identifying long-term risk patterns. When trained on both, the network ignores the risk patterns to focus on the tumors.
  • Global vs. Local Features: Grad-CAM visualizations revealed a fascinating "aha" moment: the Inherent Risk Model looks at broad, global patterns across the entire breast (likely related to tissue texture and density). In contrast, the Cancer Signs Model produces sharp, localized "hotspots" on tumor-like structures.
  • The Conflated Model is a "Weakened Hybrid": Even though the conflated model had access to the most data, it performed worse than the specialized models in their respective domains. It wasn't as good at detection as the "Signs" model, and it wasn't as good at long-term prediction as the "Inherent" model.

Key Results

  • Short-term (<1 year): The Cancer Signs model and the Conflated model performed best (AUC ~0.71-0.72), as they were essentially acting as tumor detectors.
  • Long-term (>5 years): The Inherent Risk Model was the clear winner. Its performance remained stable (AUC ~0.61-0.62) even for predictions many years out.
  • Stability: The Inherent Risk model’s AUC remained constant regardless of how close the patient was to diagnosis. This suggests it has successfully captured a "biological baseline" of risk that doesn't change just because a tumor starts to grow.
  • Clinical Impact: All AI models significantly outperformed the traditional mammographic density baseline (which hovered around an AUC of 0.54-0.55 for long-term risk).

Limitations and Future Work

The authors acknowledge a few key limitations:

  • Arbitrary Cutoff: The 1-year cutoff used to separate "risk" from "signs" is somewhat arbitrary. In reality, the transition from a high-risk environment to a detectable tumor is a biological continuum.
  • Single Modality: The study focused purely on mammograms. Future work could integrate clinical data (genetics, family history) to see if the "inherent risk" signal can be further sharpened.
  • Black Box Nature: While Grad-CAM helps, we still don't fully understand the specific biological features the "Inherent Risk" model is picking up on. Identifying these could lead to new biological insights into how breast cancer begins.

Conclusion: This paper serves as a vital reminder for AI researchers in medicine: how you curate your "positive" samples is just as important as how many you have. To predict the future, you must sometimes ignore the present.

Summary updated

Abstract

The ability to accurately estimate risk of developing breast cancer would be invaluable for clinical decision-making. One promising new approach is to integrate image-based risk models based on deep neural networks. However, one must take care when using such models, as selection of training data influences the patterns the network will learn to identify. With this in mind, we trained networks using three different criteria to select the positive training data (i.e. images from patients that will develop cancer): an inherent risk model trained on images with no visible signs of cancer, a cancer signs model trained on images containing cancer or early signs of cancer, and a conflated model trained on all images from patients with a cancer diagnosis. We find that these three models learn distinctive features that focus on different patterns, which translates to contrasts in performance. Short-term risk is best estimated by the cancer signs model, whilst long-term risk is best estimated by the inherent risk model. Carelessly training with all images conflates inherent risk with early cancer signs, and yields sub-optimal estimates in both regimes. As a consequence, conflated models may lead physicians to recommend preventative action when early cancer signs are already visible.
BibTeX
@inproceedings{liu2020decoupling,
  author = {Yue Liu and Hossein Azizpour and Fredrik Strand and Kevin Smith},
  title = {{Decoupling Inherent Risk and Early Cancer Signs in Image-Based Breast Cancer Risk Models}},
  booktitle = {International Conference on Medical Image Computing and Computer-Assisted Intervention},
  year = {2020},
  url = {https://openreview.net/forum?id=HGz-42eaqxc},
}

Loading related content...

Figure