A Structural Causal Model for MR Images of Multiple Sclerosis

Jacob C. Reinhold, Aaron Carass, Jerry L. Prince

Introduction

In precision medicine, understanding how a specific patient might respond to an intervention is a primary goal. While standard machine learning models excel at identifying correlations, they often fail to capture the underlying causal mechanisms, potentially relying on spurious correlations like scanner noise or demographic factors. This is particularly relevant in Multiple Sclerosis (MS), where the "clinico-radiological paradox" describes the moderate correlation between visible MRI lesions and a patient's actual disability.

To address this, researchers have developed a Structural Causal Model (SCM) capable of generating "counterfactual" brain MRIs. This allows for "what-if" scenarios—such as visualizing a patient's brain if their lesion volume were reduced to zero—providing a principled tool for simulating interventions in high-dimensional medical imaging.

Methodology: Causal Graphs and Hierarchical VAEs

The proposed SCM encodes functional relationships between demographic data (age, sex), disease markers (lesion volume, ventricle volume, disability scores), and the MR image itself using a directed acyclic graph (DAG). The model employs the "Abduction-Action-Prediction" framework to generate counterfactuals.

The proposed hierarchical binary latent VAE architecture, showing the flow from input to latent spaces and reconstruction.

A key challenge in this work was scaling Variational Autoencoders (VAEs) to high-resolution images ($224 \times 224$) without the model simply memorizing pixels. The authors introduced a novel architecture featuring a hierarchy of binary latent spaces ($z_0, z_1$) alongside a standard continuous latent space ($z_2$). This structure captures the rough structural outline of the brain while allowing for the manipulation of specific features. Non-linear relationships between non-image variables were modeled using Normalizing Flows (NFs), and the entire system was implemented in Pyro, a deep universal probabilistic programming language.

Experimental Setup and Datasets

The study utilized a private dataset of 171 unique scans from 77 subjects, including both healthy controls and MS patients (primarily relapsing-remitting MS). The data included $T_1$-weighted and FLAIR images. Preprocessing involved bias-field correction, rigid registration to a standard atlas, and normalization. For the model, axial slices were extracted and processed as 2D images.

Results: Counterfactual Generation and Lesion Filling

The model demonstrates the ability to generate specific counterfactual versions of existing brains rather than just realistic "average" brains. A primary application is "lesion filling" (making a diseased brain appear healthy). By setting the "lesion volume" variable to zero within the causal model, the system generates an image where hyper-intense MS lesions disappear while the overall brain structure remains preserved.

A generated counterfactual MRI slice demonstrating the model's ability to preserve structure during intervention.

Quantitative validation using a separate lesion segmentation algorithm confirmed that intervening to set lesion volume to 0 mL resulted in images with near-zero segmented lesion volume. However, the model showed "asymmetric success": while it effectively removed lesions, it was less consistent at adding large lesion loads to healthy brains, likely due to the limitations of 2D slice processing.

Discussion and Future Work

While the SCM provides a powerful framework for retrospective analysis and controlling for confounders, several challenges remain. Counterfactuals are inherently unverifiable since the "ground truth" parallel universe does not exist. Furthermore, MS lesions are 3D structures; moving from 2D slices to a full 3D SCM is a critical next step for improving consistency. Future iterations will also aim to include more variables, such as specific treatment types, to better capture the complexity of MS progression.

Summary updated

Abstract

Precision medicine involves answering counterfactual questions such as “Would this patient respond better to treatment A or treatment B?” These types of questions are causal in nature and require the tools of causal inference to be answered, e.g., with a structural causal model (SCM). In this work, we develop an SCM that models the interaction between demographic information, disease covariates, and magnetic resonance (MR) images of the brain for people with multiple sclerosis. Inference in the SCM generates counterfactual images that show what an MR image of the brain would look like if demographic or disease covariates are changed. These images can be used for modeling disease progression or used for downstream image processing tasks where controlling for confounders is necessary.
BibTeX
@inproceedings{reinhold2021structural,
  author = {Jacob C. Reinhold and Aaron Carass and Jerry L. Prince},
  title = {{A Structural Causal Model for MR Images of Multiple Sclerosis}},
  booktitle = {International Conference on Medical Image Computing and Computer-Assisted Intervention},
  year = {2021},
  doi = {10.1007/978-3-030-87240-3\_75},
}
Figure