Causal Representation Learning from Multimodal Biomedical Observations

Yuewen Sun, Lingjing Kong, Guangyi Chen, Loka Li, Gongxu Luo, Zijian Li, Yixuan Zhang, Yujia Zheng, Mengyue Yang, Petar Stojanov, Eran Segal, Eric P. Xing, Kun Zhang

TL;DR

This paper introduces a new framework for Causal Representation Learning (CRL) that can pinpoint individual, interpretable causal factors from complex, multimodal biomedical data. By leveraging the natural "sparsity" of biological connections—where one physiological factor typically only influences a few others—the researchers provide the first theoretical guarantees for identifying specific latent variables without relying on overly restrictive mathematical assumptions.

What Problem Does This Paper Address?

In biomedical research, we are often drowning in multimodal data: a single patient might have genetic sequences, chest X-rays, and sleep monitor logs. While deep learning models are great at predicting outcomes from this data, they are usually "black boxes." They lack identifiability, meaning there is no guarantee that the "features" the model learns actually correspond to real-world biological mechanisms like lung density or heart rate variability.

Existing causal models usually fall into two traps:

  1. Too Coarse: They can only identify "subspaces" (groups of variables) rather than individual factors.
  2. Too Rigid: They assume the data follows specific, simple mathematical distributions (like the exponential family) that rarely hold true in the messy world of biology.

This paper bridges the gap by providing a way to identify individual causal components in a nonparametric setting—meaning the model is flexible enough to handle the complex, nonlinear relationships inherent in human physiology.

Methods

The authors propose a two-step theoretical approach, instantiated in a practical deep learning architecture:

  1. Subspace Identification: They first prove that you can separate modality-specific "noise" from the meaningful latent variables, provided those variables exert enough influence on other modalities.
  2. Component-wise Disentanglement: This is the "secret sauce." They leverage structural sparsity. In biological systems, causal graphs are rarely fully connected. By enforcing a sparsity constraint (using an $L_1$ penalty), the model learns that any "mixing" of latent variables within a modality would create an unnecessarily dense and complex web of cross-modality connections. The simplest (sparsest) explanation that fits the data is usually the true causal one.
The proposed estimation framework uses encoders to extract latents ($z$) and domain-specific info ($\eta$), an adjacency matrix to model causal links, and normalizing flows to handle the noise.

The implementation uses an encoder-decoder framework combined with normalizing flows to estimate the exogenous noise terms and a learnable adjacency matrix to map out the causal structure.

Datasets

The authors validated their approach across three distinct types of data:

  • Numerical Simulations: Synthetic datasets with varying numbers of modalities and sparsity ratios to test the limits of their theoretical guarantees.
  • Variant MNIST: A clever synthetic image dataset where digit transparency in one modality (Colored MNIST) causally influences the vertical position of items in another (Fashion MNIST).
  • Human Phenotype Dataset: A massive real-world dataset containing 30+ modalities, including blood tests, anthropometry, and fundus (eye) imaging.

Key Insights and Ideas

  • Sparsity as a Compass: The most important insight is that "mixing" latent variables makes the world look more complicated than it is. If the model incorrectly blends two independent biological factors, it will suddenly appear as if those factors have many more causal links to other modalities than they actually do. See Figure 3 for a visual of how an "alternative" (incorrect) graph becomes denser than the "true" graph.
  • Moving Beyond "Shared" Variables: Unlike previous work that required variables to be perfectly shared across modalities, this model only requires that they exert "sufficient influence." This is much more realistic for medicine, where a genetic marker might influence a sleep pattern without being part of the sleep data itself.
  • Nonparametric Flexibility: By avoiding parametric assumptions (like assuming everything is Gaussian), the model can capture the highly nonlinear interactions common in fMRI or gene regulatory networks.
A comparison of how this work stacks up against previous CRL methods, showing its unique ability to handle multiple modalities, nonparametric distributions, and component-wise identification.

Key Results

  • High Fidelity Recovery: In numerical experiments, the model achieved a Mean Correlation Coefficient (MCC) near 1.0, indicating near-perfect recovery of the true latent variables, significantly outperforming baselines like BetaVAE and CausalVAE.
  • Sparsity Validation: The ablation studies confirmed that as the underlying causal graph becomes sparser, the model’s ability to identify the correct variables improves, validating their core theoretical contribution.
  • Real-World Discovery: On the Human Phenotype Dataset, the model discovered causal links that align with medical literature. For example, it identified a latent sleep factor that causally influences oxygen saturation—a known relationship in obstructive sleep apnea research. It also found a link between fundus (eye) images and handgrip strength, a connection recently highlighted in aging research.

Limitations and Future Work

While powerful, the framework currently requires the user to pre-define the number of latent variables in each modality, which isn't always known in exploratory research. The authors suggest that cross-validation can help mitigate this, but a more automated way to discover the "latent dimension" would be a significant upgrade. Additionally, while the model is theoretically sound, the authors note that more extensive benchmarking against specialized biomedical models is needed to prove its utility in specific clinical diagnostic pipelines.

Summary updated

Abstract

Prevalent in biomedical applications (e.g., human phenotype research), multimodal datasets can provide valuable insights into the underlying physiological mechanisms. However, current machine learning (ML) models designed to analyze these datasets often lack interpretability and identifiability guarantees, which are essential for biomedical research. Recent advances in causal representation learning have shown promise in identifying interpretable latent causal variables with formal theoretical guarantees. Unfortunately, most current work on multimodal distributions either relies on restrictive parametric assumptions or yields only coarse identification results, limiting their applicability to biomedical research that favors a detailed understanding of the mechanisms. In this work, we aim to develop flexible identification conditions for multimodal data and principled methods to facilitate the understanding of biomedical datasets. Theoretically, we consider a nonparametric latent distribution (c.f., parametric assumptions in previous work) that allows for causal relationships across potentially different modalities. We establish identifiability guarantees for each latent component, extending the subspace identification results from previous work. Our key theoretical contribution is the structural sparsity of causal connections between modalities, which, as we will discuss, is natural for a large collection of biomedical systems. Empirically, we present a practical framework to instantiate our theoretical insights. We demonstrate the effectiveness of our approach through extensive experiments on both numerical and synthetic datasets. Results on a real-world human phenotype dataset are consistent with established biomedical research, validating our theoretical and methodological framework.
BibTeX
@inproceedings{sun2025causal,
  author = {Yuewen Sun and Lingjing Kong and Guangyi Chen and Loka Li and Gongxu Luo and Zijian Li and Yixuan Zhang and Yujia Zheng and Mengyue Yang and Petar Stojanov and Eran Segal and Eric P. Xing and Kun Zhang},
  title = {{Causal Representation Learning from Multimodal Biomedical Observations}},
  booktitle = {International Conference on Learning Representations},
  year = {2025},
  url = {https://openreview.net/forum?id=hjROBHstZ3},
}
Figure