DBCE: A Saliency Method for Medical Deep Learning Through Anatomically-Consistent Free-Form Deformations

Joshua Peters, Léo Lebrat, Rodrigo Santa Cruz, Aaron Nicolson, Gregg Belous, Salamata Konate, Parnesh Raniga, Vincent Dore, Pierrick Bourgeat, Jurgen Mejan-Fripp, Clinton Fookes, Olivier SalvadoWACV 2023

TL;DR

This paper introduces DBCE, a novel method for generating anatomically consistent saliency maps for medical deep learning models. By optimizing free-form deformations to flip a model's prediction, DBCE provides intuitive visualizations of tissue atrophy and expansion, offering a more interpretable and clinically relevant explanation than traditional saliency methods.

What Problem Does This Paper Address?

Deep learning (DL) models are increasingly used in medical imaging for tasks like disease detection. However, their "black-box" nature makes it difficult for clinicians to understand why a model makes a particular prediction. This lack of interpretability hinders trust and adoption in healthcare. Existing saliency methods, which highlight important image regions, often produce scattered or overly broad maps, leading to potential misinterpretation or automation bias. Furthermore, many methods fail rigorous "sanity checks" designed to assess their reliability and repeatability. This paper aims to bridge this interpretability gap by providing more meaningful, anatomically grounded saliency maps.

Methods

DBCE (Deformation Based Counterfactual Explainability) is a generation-based method that works by optimizing a smooth, anatomically plausible deformation of an input medical image. The goal is to create a "counterfactual" image that, when fed into a trained DL model, flips its prediction. This deformation is achieved using Free-Form Deformations (FFD), a technique that parameterizes smooth, invertible transformations.

The core of DBCE involves solving an optimization problem: given an input image $x$ and a trained DL model $f_\theta$, DBCE finds a deformation $T_\Phi$ that transforms $x$ into $x̃$ such that $f_\theta(x̃)$ has the opposite sign of $f_\theta(x)$. This optimization is guided by a loss function that encourages the prediction flip while also incorporating regularization terms. These regularization terms are crucial for ensuring anatomical plausibility:

  • Local Invertibility: A penalty term prevents the deformation from creating foldings or singularities, preserving the image's structure.
  • Sparsity: An L1-regularization term encourages the deformation to be sparse, meaning it affects only specific regions.
  • Localization: A third term allows for constraining the deformation to a specific anatomical region of interest.

The saliency map is then derived from the norm of the resulting deformation, and a more intuitive visualization is obtained by showing the difference between the original and the deformed image.

Datasets

The paper uses a three-dimensional T1w structural MRI dataset for Alzheimer's Disease (AD) detection. The dataset is an augmented version of the Alzheimer’s Disease Neuroimaging Initiative (ADNI) dataset, with a train/validation/test split of 776/342/520 images.

Key Insights and Ideas

  • Anatomical Consistency is Key: Traditional saliency methods often ignore anatomical constraints. DBCE's use of diffeomorphic deformations ensures that the generated explanations are anatomically plausible, making them more interpretable for clinicians.
  • Flipping Predictions Reveals Importance: By finding the minimal deformation needed to change a model's prediction, DBCE effectively highlights the image regions that are most critical for that decision. This is a more direct way of understanding the model's reasoning than simply highlighting existing features.
  • DBCE Passes Sanity Checks: The method demonstrates good repeatability and is less susceptible to model initialization variations compared to classical sensitivity-based methods, addressing a critical need for trustworthy AI in healthcare.
  • Localised Deformations Reveal Model Robustness: The experiment on localized deformations suggests that the lightweight CNN used in their AD detection task relies on a combination of spatially disjoint features, making it robust to small, localized changes.

Key Results

  • DBCE produces saliency maps that are more meaningful and anatomically consistent compared to Gradient, DeconvNet, and GradCAM.
  • For a cohort of patients, DBCE highlights temporal lobes as discriminating regions for AD diagnosis, aligning with existing medical knowledge.
  • The method achieves good intra-architecture repeatability, with a structural similarity index greater than 0.5 (0.745 ± 0.039).
  • The average difference between original and deformed images across a cohort of patients localizes atrophy to temporal and ventricle regions, consistent with AD progression.
  • The optimization process for DBCE is computationally affordable, converging in about 8.45 seconds per gradient step on a GPU.

Limitations and Future Work

  • Ill-posedness for Single Anatomical Zones: The paper notes that while DBCE can be localized, forcing deformations on a single anatomical zone can sometimes lead to issues, suggesting that the model's decision might rely on a combination of features.
  • Future Work: The authors suggest exploring additional visualization techniques and further investigating the robustness of DBCE to additive Gaussian noise, which they provide in supplementary materials. They also highlight the potential for DBCE to be used with other DL architectures beyond CNNs.

Summary updated

BibTeX
@inproceedings{peters2023dbce,
  author = {Joshua Peters and Léo Lebrat and Rodrigo Santa Cruz and Aaron Nicolson and Gregg Belous and Salamata Konate and Parnesh Raniga and Vincent Dore and Pierrick Bourgeat and Jurgen Mejan-Fripp and Clinton Fookes and Olivier Salvado},
  title = {{DBCE: A Saliency Method for Medical Deep Learning Through Anatomically-Consistent Free-Form Deformations}},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
  year = {2023},
}
Figure