Identifiability Guarantees for Causal Disentanglement from Soft Interventions

Jiaqi Zhang, Chandler Squires, Kristjan H. Greenewald, Akash Srivastava, Karthikeyan Shanmugam, Caroline Uhler

Introduction

In fields ranging from genomics to computer vision, researchers are often confronted with high-dimensional "ambient" data—such as gene expression profiles or pixel arrays—that are generated by a smaller set of underlying causal variables. The task of mapping this complex, observed data back to its latent causal drivers and their interactions is known as causal disentanglement.

A primary hurdle in this field is identifiability: the mathematical guarantee that the true causal model can be uniquely recovered from the observed data. Without restrictive assumptions, many different causal structures can produce identical data distributions. While previous research suggested that "hard" interventions (where a variable is fixed to a specific value) or "paired" data (observations of the same unit before and after a change) could resolve this ambiguity, such data are often difficult or impossible to collect in real-world settings.

This paper addresses whether a latent causal model can be uniquely identified using only "soft" interventions—perturbations that shift a variable’s distribution without completely overriding its natural mechanism—applied to "unpaired" data.

Theoretical Framework: Linear Interventional Faithfulness

The authors establish the first theoretical guarantees for identifiability in this setting. Central to their proof is a novel condition termed linear interventional faithfulness.

In high-dimensional settings, latent causal variables are often mixed together through complex, non-linear functions to produce the observed data. Linear interventional faithfulness ensures that the effects of an intervention on a latent variable are not "canceled out" or rendered invisible during this mixing process. Under this assumption, the authors prove that the latent causal model is identifiable up to a CD-equivalence class, which means the variables and their relationships can be recovered up to a permutation and scaling.

Furthermore, the research demonstrates that the true causal graph is the sparsest one capable of explaining the variations observed across different interventional datasets. This provides a clear objective for optimization: searching for the simplest model that fits the data.

The CausalDiscrepancyVAE

To translate these theoretical insights into a practical tool, the authors developed the CausalDiscrepancyVAE. This architecture consists of three primary components:

  1. Encoder: Maps the high-dimensional observations ($X$) to exogenous noise variables ($Z$).
  2. Causal Decoder: Utilizes a Deep Structural Causal Model (DSCM) to transform the noise ($Z$) into latent causal variables ($U$) based on a learnable Directed Acyclic Graph (DAG).
  3. Mixing Function: A deep neural network that maps the latent variables ($U$) back into the high-dimensional observation space.

The model is trained using a specialized Discrepancy Loss based on Maximum Mean Discrepancy (MMD). Unlike standard Variational Autoencoders that focus primarily on reconstruction, this loss term compares "virtual" counterfactuals—generated by the model’s internal causal logic—against actual samples from the interventional distributions. This forces the model to learn the true underlying mechanisms rather than just statistical correlations.

Experimental Results and Biological Discovery

The framework was rigorously tested on both synthetic datasets and real-world biological data, specifically focusing on single-cell genomics.

Perturb-seq Analysis

The primary testbed was a Perturb-seq dataset, which includes single-cell RNA sequencing of human cells. The dataset contains approximately 9,000 unperturbed cells and 100,000 cells where specific genes were activated using CRISPR.

The CausalDiscrepancyVAE demonstrated remarkable performance:

  • High Predictive Accuracy: On the Perturb-seq dataset, the model achieved an average $R^2$ of 0.99 for predicting the effects of single-gene perturbations.
  • Zero-Shot Combinatorial Extrapolation: Most impressively, the model could predict the effects of double-gene perturbations (which were entirely withheld during training) with an $R^2$ of 0.98. This suggests the model truly captured the underlying biological "programs" rather than just memorizing patterns.

Biological Insights

Beyond numerical accuracy, the model recovered known regulatory pathways. For instance, it correctly identified the relationship between the gene DUSP9 and its downstream targets MAPK1 and ETS2, validating the model's utility for biological discovery.

Limitations and Future Directions

While this work represents a significant step forward, several challenges remain:

  • Intervention Coverage: The current theory assumes an intervention is available for every latent node, which may not always be feasible in drug discovery or complex systems.
  • Measurement Noise: The proofs assume noiseless observations; extending these to noisy, real-world measurement models is a priority for future research.
  • Optimization Complexity: Learning a DAG is a combinatorially hard problem. The gradient-based relaxation used here is effective but can be sensitive to initialization and requires careful hyperparameter tuning.

Conclusion

By proving that soft interventions are sufficient for causal disentanglement, this paper opens new avenues for understanding complex systems. The ability to predict combinatorial effects without direct observation suggests that we can move from merely observing high-dimensional data to truly understanding the causal mechanisms that drive it.

Summary updated

BibTeX
@inproceedings{zhang2023identifiability,
  author = {Jiaqi Zhang and Chandler Squires and Kristjan H. Greenewald and Akash Srivastava and Karthikeyan Shanmugam and Caroline Uhler},
  title = {{Identifiability Guarantees for Causal Disentanglement from Soft Interventions}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2023},
  url = {https://openreview.net/forum?id=o16sYKHk3S},
}
Figure