Multimodal explainable artificial intelligence identifies patients with non-ischaemic cardiomyopathy at risk of lethal ventricular arrhythmias

Maarten Z H Kolk, Samuel Ruipérez-Campillo, Cornelis P. Allaart, Arthur A.m. Wilde, Reinoud E. Knops, Sanjiv M. Narayan, Fleur V.y. Tjong, Femke D. Raijmakers, Anne‐Lotte C.j. van der Lingen, Marco Götte, Jasper L. Selder, Laura Alvarez-Florez, Ivana Išgum, Erik J. BekkersScientific 2024 · DOI

TL;DR

Researchers have developed DEEP RISK, a multimodal AI framework that predicts the risk of lethal heart rhythms in patients with non-ischaemic cardiomyopathy (NICM). By combining deep learning features from cardiac MRIs and ECGs with standard clinical data, the model significantly outperforms current clinical guidelines, potentially reducing unnecessary heart implant surgeries while ensuring high-risk patients receive life-saving care.

What Problem Does This Paper Address?

Sudden cardiac death (SCD) caused by malignant ventricular arrhythmias is a major killer. For years, the standard "shield" against this has been the Implantable Cardioverter-Defibrillator (ICD). However, while ICDs are clearly effective for patients who have suffered heart attacks (ischaemic cases), their benefit for patients with non-ischaemic cardiomyopathy (NICM)—where heart failure isn't caused by clogged arteries—is hotly debated.

Currently, doctors decide who gets an ICD based largely on Left Ventricular Ejection Fraction (LVEF), a measure of how much blood the heart pumps out. But LVEF is a blunt instrument; many patients with low LVEF never experience a dangerous arrhythmia, while some with "safe" LVEF do. This leads to both "unnecessary" implants (which carry risks of infection and lead failure) and missed opportunities to save lives. The challenge is to find a more precise, personalized way to stratify risk.

Methods

The authors developed a pipeline called DEEP RISK that moves away from "hand-crafted" clinical features toward deep-learned representations.

  1. Feature Extraction (The VAEs): Instead of just looking at scar size or QRS duration, the team used Residual Variational Autoencoders (VAEs). These neural networks were trained on large datasets to compress complex 3D LGE-MRI scans (which show heart scarring) and 12-lead ECG signals into low-dimensional "latent representations." This allows the model to capture subtle patterns in heart geometry and electrical activity that the human eye might miss.
  2. The Predictor (XGBoost): These latent features, along with 21 clinical variables (like age and medication), were fed into an XGBoost classifier to predict the probability of a malignant arrhythmia within one year.
  3. Explainability (XAI): To avoid the "black box" problem, the researchers used a three-tiered approach:
  4. SHAP values to rank which features (MRI vs. ECG vs. Clinical) were most important.
  5. Latent Traversal to visualize what a specific "feature" actually looks like (e.g., how a latent variable corresponds to a specific area of heart fibrosis).
  6. Gradient-based attention maps to highlight exactly which regions of a specific patient's MRI or ECG triggered the high-risk warning.

Datasets

The study used a retrospective cohort of 289 NICM patients from two tertiary hospitals in the Netherlands:

  • Development Cohort: 186 patients (Hospital A).
  • External Validation Cohort: 103 patients (Hospital B).

The VAEs themselves were pre-trained on much larger datasets (over 333,000 ECGs and 970 MRI scans) to ensure they could robustly "understand" the underlying physiology before being applied to the specific NICM risk task.

Key Insights and Ideas

  • Multimodality is Non-Negotiable: The model performed best when it had access to MRI, ECG, and clinical data. While the MRI branch was the strongest individual predictor, adding electrical (ECG) and clinical data provided "complementary" information about the heart's substrate and its triggers.
  • MRI as the "Substrate" King: The LGE-MRI features were far more predictive than ECG or clinical data alone. This reinforces the idea that the physical distribution of fibrosis (scarring) is the primary driver of arrhythmia risk in NICM.
  • High Sensitivity is the Goal: In a clinical setting, missing a lethal arrhythmia is a catastrophe. DEEP RISK achieved a sensitivity of 0.98, meaning it caught almost every patient who actually needed an ICD, while still maintaining enough specificity to potentially reduce "unnecessary" implants by a significant margin.

Key Results

  • Predictive Power: The multimodal DEEP RISK model achieved an AUROC of 0.84 in the external validation cohort.
  • Comparison: It significantly outperformed individual branches:
  • MRI only: 0.80 AUROC
    • Clinical only: 0.64 AUROC
    • ECG only: 0.54 AUROC
  • Clinical Impact: In the validation group of 103 patients, the model correctly identified all 6 patients who experienced a malignant arrhythmia. It also correctly identified 71 patients who did not have an event, potentially sparing them from unnecessary procedures, though it did suggest 26 "unnecessary" implants (false positives).

Limitations and Future Work

  • The "Appropriate Therapy" Proxy: The study uses "ICD shock/ATP" as a surrogate for sudden death. However, some arrhythmias treated by an ICD might have stopped on their own (self-terminated), meaning the model might slightly overestimate the life-saving benefit.
  • Sample Size: While 289 patients is a solid start for this niche, deep learning models generally thrive on much larger, more diverse datasets to ensure they generalize across different global populations and ethnicities.
  • Static vs. Dynamic: The model uses a "snapshot" of data from before the implant. Future work could involve longitudinal data—tracking how a patient's risk changes over time as their heart failure progresses or improves with medication.

Summary updated

Abstract

The efficacy of an implantable cardioverter-defibrillator (ICD) in patients with a non-ischaemic cardiomyopathy for primary prevention of sudden cardiac death is increasingly debated. We developed a multimodal deep learning model for arrhythmic risk prediction that integrated late gadolinium enhanced (LGE) cardiac magnetic resonance imaging (MRI), electrocardiography (ECG) and clinical data. Short-axis LGE-MRI scans and 12-lead ECGs were retrospectively collected from a cohort of 289 patients prior to ICD implantation, across two tertiary hospitals. A residual variational autoencoder was developed to extract physiological features from LGE-MRI and ECG, and used as inputs for a machine learning model (DEEP RISK) to predict malignant ventricular arrhythmia onset. In the validation cohort, the multimodal DEEP RISK model predicted malignant ventricular arrhythmias with an area under the receiver operating characteristic curve (AUROC) of 0.84 (95% confidence interval (CI) 0.71-0.96), a sensitivity of 0.98 (95% CI 0.75-1.00) and a specificity of 0.73 (95% CI 0.58-0.97). The models trained on individual modalities exhibited lower AUROC values compared to DEEP RISK [MRI branch: 0.80 (95% CI 0.65-0.94), ECG branch: 0.54 (95% CI 0.26-0.82), Clinical branch: 0.64 (95% CI 0.39-0.87)]. These results suggest that a multimodal model achieves high prognostic accuracy in predicting ventricular arrhythmias in a cohort of patients with non-ischaemic systolic heart failure, using data collected prior to ICD implantation.
BibTeX
@inproceedings{kolk2024multimodal,
  author = {Maarten Z H Kolk and Samuel Ruipérez-Campillo and Cornelis P. Allaart and Arthur A.M. Wilde and Reinoud E. Knops and Sanjiv M. Narayan and Fleur V.Y. Tjong and Femke D. Raijmakers and Anne‐Lotte C.J. van der Lingen and Marco Götte and Jasper L. Selder and Laura Alvarez-Florez and Ivana Išgum and Erik J. Bekkers and Erik J. Bekkers},
  title = {{Multimodal explainable artificial intelligence identifies patients with non-ischaemic cardiomyopathy at risk of lethal ventricular arrhythmias}},
  booktitle = {Scientific},
  year = {2024},
  doi = {10.1038/s41598-024-65357-x},
}
Figure