Interpretability of a deep learning model in the application of cardiac MRI segmentation with an ACDC challenge dataset

Adrianna Janik, Jonathan Dodd, Georgiana Ifrim, Kris Sankaran, Kathleen M. Curran

TL;DR

This paper explores the interpretability of deep learning models used for cardiac MRI segmentation, a crucial task for diagnosing heart conditions. By applying the D-TCAV method, they aim to understand why a model segments the heart's chambers and muscle, moving beyond just how well it performs. This research is vital for building trust and enabling clinical adoption of AI in cardiology.

What Problem Does This Paper Address?

Cardiovascular disease is a leading cause of death globally, and Cardiac Magnetic Resonance (CMR) imaging is the gold standard for assessing heart conditions like cardiomyopathy. While deep learning (DL) has revolutionized medical image analysis, including cardiac MRI segmentation, these "black-box" models lack transparency. This opacity is a significant barrier to their widespread clinical adoption, especially with regulations like GDPR emphasizing explainability. Radiologists and cardiologists need to understand why a model makes a particular prediction or segmentation to trust and utilize it effectively. This paper addresses this gap by investigating methods to interpret DL models used for segmenting key cardiac structures (Left Ventricle, Right Ventricle, and Myocardium) from CMR images.

Methods

The core of this research lies in applying and adapting the Discovering and Testing with Concept Activation Vectors (D-TCAV) method for semantic segmentation tasks. D-TCAV is an extension of the original TCAV method, designed to provide interpretable explanations for DL model predictions.

Here's a breakdown of the D-TCAV process as applied in this study:

  1. Super-pixel Generation: The input cardiac MRI images are first segmented into "super-pixels" using the SLIC algorithm. Super-pixels are groups of pixels with similar properties, assumed to capture local perceptual meaning.
  2. Super-pixel Embeddings: These super-pixels are then fed into the trained deep learning model (a U-Net in this case). Activations from a middle layer of the network are collected, representing the "embeddings" or latent space representation of these super-pixels.
  3. Clustering and Concept Discovery: The latent space embeddings are clustered. This clustering aims to group similar representations, and these clusters are considered potential "concepts" that the model has learned. Outliers are removed, and dimensionality reduction (UMAP) is used to aid clustering.
  4. Concept Activation Vectors (CAVs) and TCAV: For each identified concept (cluster), a CAV is generated. This vector represents the direction in the latent space that corresponds to the concept. The TCAV score is then calculated to quantify how important a particular concept is for predicting a specific cardiac structure or pathology. The D-TCAV method's advantage is that it automates concept discovery, unlike the original TCAV which required manual concept definition.

The segmentation model itself is a U-Net architecture, a standard for image segmentation tasks, trained to identify the Left Ventricle (LV), Right Ventricle (RV), and Myocardium (MYO).

Datasets

The study utilizes the MICCAI 2017 challenge dataset for cardiac MRI. This dataset comprises images from 100 patients, divided into five subgroups: one healthy control group and four cardiomyopathy groups (chronic myocardial infarction, dilated cardiomyopathy, hypertrophic cardiomyopathy, and dilated right ventricle). The data includes both end-systole and end-diastole phases. The dataset is split into training (70 patients) and development (30 patients) sets.

Key Insights and Ideas

  • D-TCAV for Segmentation is Novel: Applying D-TCAV, an interpretability method originally designed for classification, to a semantic segmentation task (pixel-level classification) is a novel contribution. This opens up new avenues for understanding complex segmentation models.
  • Concepts Reflect Anatomical Structures, Not Just Pathology: The discovered concepts were found to be important across various cardiac pathology classes. This suggests that the U-Net model, trained for segmentation, primarily learned to represent the anatomical structures (LV, RV, Myocardium) rather than specific disease characteristics.
  • Need for Explicit Feature Extraction: The findings imply that for models to learn disease-specific features, specialized training objectives or post-segmentation feature engineering might be necessary. The current segmentation-focused training doesn't inherently capture pathological nuances.
  • Reduced Pre-processing Time: The D-TCAV method, by automating concept discovery, offers a shorter pre-processing time for clinicians compared to methods requiring manual concept definition.

Key Results

  • Segmentation Performance: The U-Net models achieved state-of-the-art performance on the ACDC dataset, with Dice scores for the trained models reaching high levels (e.g., global Dice score of 91% for the basic U-Net on the training set and 61% on the development set). The study reports Dice scores for LV, RV, and MYO segmentation.
  • Concept Identification: The D-TCAV method successfully identified clusters of super-pixels that could be considered representations of cardiac structures.
  • TCAV Scores: Statistically significant TCAV scores were obtained for several concepts. For instance, one concept (cluster 56) showed a high score for the MINF (myocardial infarction) class (0.49), with lower scores for other pathologies. However, the study notes that TCAV scores for different pathologies and the control group often lie close to each other, reinforcing the idea that concepts represent anatomy more than specific conditions.

Limitations and Future Work

  • Anatomical vs. Pathological Concepts: A key limitation identified is that the learned concepts primarily represent anatomical structures (LV, RV, Myocardium) rather than specific cardiac conditions. This suggests that the segmentation objective alone does not capture disease-specific information.
  • Need for Clinical Validation: The authors emphasize the need for further clinical validation of the discovered concepts with expert radiologists to assess their true clinical interpretability and utility.
  • Hierarchical Clustering: The paper suggests that exploring hierarchical clustering or other methods for discovering more localized concepts might yield better results.
  • Automated Heart Localization: While the study used ground-truth masks for cropping the region of interest, an automated heart localization method could improve robustness for unseen data.
  • Future Directions: Future work could involve designing training objectives that explicitly incorporate disease information to learn more pathology-specific concepts, or exploring post-segmentation feature extraction techniques. A validation study in a clinical setting is also recommended.

Summary updated

Abstract

Cardiac Magnetic Resonance (CMR) is the most effective tool for the assessment and diagnosis of a heart condition, which malfunction is the world's leading cause of death. Software tools leveraging Artificial Intelligence already enhance radiologists and cardiologists in heart condition assessment but their lack of transparency is a problem. This project investigates if it is possible to discover concepts representative for different cardiac conditions from the deep network trained to segment crdiac structures: Left Ventricle (LV), Right Ventricle (RV) and Myocardium (MYO), using explainability methods that enhances classification system by providing the score-based values of qualitative concepts, along with the key performance metrics. With introduction of a need of explanations in GDPR explainability of AI systems is necessary. This study applies Discovering and Testing with Concept Activation Vectors (D-TCAV), an interpretaibilty method to extract underlying features important for cardiac disease diagnosis from MRI data. The method provides a quantitative notion of concept importance for disease classified. In previous studies, the base method is applied to the classification of cardiac disease and provides clinically meaningful explanations for the predictions of a black-box deep learning classifier. This study applies a method extending TCAV with a Discovering phase (D-TCAV) to cardiac MRI analysis. The advantage of the D-TCAV method over the base method is that it is user-independent. The contribution of this study is a novel application of the explainability method D-TCAV for cardiac MRI anlysis. D-TCAV provides a shorter pre-processing time for clinicians than the base method.
BibTeX
@inproceedings{janik2021interpretability,
  author = {Adrianna Janik and Jonathan Dodd and Georgiana Ifrim and Kris Sankaran and Kathleen M. Curran},
  title = {{Interpretability of a deep learning model in the application of cardiac MRI segmentation with an ACDC challenge dataset}},
  booktitle = {Image Processing},
  year = {2021},
  doi = {10.1117/12.2582227},
}
Figure