Quantifying Confounding Bias in Neuroimaging Datasets with Causal Inference
Introduction
As neuroimaging moves toward "big data," researchers increasingly pool MRI scans from multiple independent studies to achieve the sample sizes required for complex machine learning models. However, these datasets are collected with varying goals, demographics, inclusion criteria, and scanner hardware. This heterogeneity introduces severe biases: selection bias (e.g., recruiting specific age groups), acquisition bias (e.g., different MRI manufacturers), and processing bias. If a machine learning model is trained on this pooled data, it risks learning these biases rather than true, generalizable biological signals. Furthermore, it is difficult to determine if a statistical correlation is a true biological cause or if it is confounded by an unmeasured third variable, such as scanner type.
Methodology
The authors address this problem through a two-phase approach: detection and quantification.
Detection: "Name That Dataset"
To detect bias, the authors trained a Random Forest classifier to predict the source dataset of a scan using only FreeSurfer-derived physical measurements (brain volume and cortical thickness). In an unbiased, harmonized environment, the classifier should perform near random chance.
Quantification: Causal Inference via MDL
To systematically study bias, the authors proposed a causal inference framework to distinguish between a causal model (biological factors $X$ directly influence brain measure $Y$) and a confounded model (latent confounders $Z$ influence both $X$ and $Y$).
The framework relies on the algorithmic Markov condition, which posits that the true generative process is the one with the lowest Kolmogorov complexity. This is approximated using Minimum Description Length (MDL). The causal model is represented by linear regression, while the confounded model is represented by probabilistic Principal Component Analysis (PCA). By calculating the difference in MDL between these models, the authors can quantify the extent of confounding without identifying the specific confounder.
Datasets
The study utilized a massive pooled dataset of 12,207 MRI T1 brain scans from 15 large-scale public datasets, including ABIDE I & II, ADHD200, ADNI, HCP, and others. All scans were processed uniformly using FreeSurfer version 5.3 to extract volume and thickness metrics, with only one scan per subject retained to avoid longitudinal data leakage.
Key Results
The results highlight significant hidden biases in pooled neuroimaging data.
Dataset Classification
The Random Forest classifier achieved 73.3% accuracy in predicting the source dataset based on volume and thickness, far exceeding the random chance level of 6.7%. Even using only age and sex yielded 42.2% accuracy, proving that while demographics play a role, the high accuracy is primarily driven by the image-derived measurements themselves. This suggests that the scanner and study design leave an indelible "fingerprint" on the data.
Causal vs. Confounded Relationships
Analysis of the relationship between demographics (age, sex) and brain volume revealed:
- High Causality: The Human Connectome Project (HCP), which uses a single dedicated scanner and strict protocols, showed the strongest evidence of true causal relationships.
- High Confounding: Heterogeneous datasets (e.g., COBRE, HBN, PPMI) that pool data from multiple sites or have loose inclusion criteria showed significant confounding.
- Biological Plausibility: In the ADNI dataset, the MDL method correctly identified the expansion of lateral ventricles as highly causal—a known biological reality of aging—while flagging cerebellum cortex variations as heavily confounded.
Discussion and Limitations
While this framework successfully detects and quantifies bias, it does not yet provide a method to remove it. The results are also tied to the specific mathematical models chosen (linear regression and PCA). Furthermore, the authors fixed the dimensionality of the unobserved confounder to $k=1$, whereas real-world confounding is likely multidimensional. Future work may extend this framework to estimate the causal effects of technical parameters, such as the impact of magnetic field strength on signal-to-noise ratios.
Summary updated