Abstract
Causal representation learning (CRL) aims to recover structural causal models from high-dimensional observations. While previous work established identifiability under the assumption of atomic interventions, this requirement is often impractical. We present a framework for CRL under subset interventions, proving that latent models can be identified up to a "causal abstraction." This abstraction provides a consistent, higher-level description of the system's causal structure based on the granularity of available interventions.
Introduction: Beyond Atomic Interventions
Causal Representation Learning (CRL) seeks to uncover the latent variables and causal mechanisms that drive complex systems. A central challenge in CRL is identifiability: ensuring that the learned model reflects the true underlying causal structure. Traditional approaches often rely on "atomic" interventions—the ability to manipulate each latent variable independently.
In practice, atomic interventions are rarely possible. Biological perturbations might affect entire gene pathways, and image edits often modify multiple semantic features at once. This paper addresses the realistic scenario where only subset interventions are possible, asking what causal information remains recoverable under these constraints.
Methods: Defining Causal Abstractions
The authors utilize a weakly-supervised CRL framework, learning from counterfactual pairs $(x, \tilde{x})$—observations of a system before and after an unknown intervention. To handle subset interventions, they introduce the concept of Causal Abstractions.
Instead of attempting to recover the exact micro-level graph, the framework identifies a "macro-level" version where nodes are grouped into blocks. This is achieved by leveraging Non-Descendant Invariance: the principle that when a subset of variables is intervened upon, their ancestors and unrelated nodes remain unchanged. By analyzing the overlapping sets of invariant variables across different interventions, the model constructs a "quotient graph" representing the highest level of granularity uniquely identifiable from the data.
Key Theoretical Results
The paper establishes two primary theorems that redefine the boundaries of identifiability:
- Theorem 3.1 (SCM Abstraction): Any latent causal model is identifiable up to an SCM Abstraction. The resulting causal graph is the quotient graph of the true graph with respect to the partition generated by the non-descendant sets of the available interventions.
- Theorem 3.2 (Deep Disentanglement): Even when causal mechanisms are obscured, individual variables can often be disentangled if the intervention patterns are sufficiently diverse.
Experimental Validation
The theory was validated using synthetic linear Gaussian models. Latent variables followed a Directed Acyclic Graph (DAG) and were mixed into high-dimensional space using random rotation matrices. Experiments confirmed that the learned encoders successfully recovered the block-diagonal structure predicted by the causal abstraction theory, demonstrating that identifiability is a spectrum determined by intervention diversity.
Conclusion and Future Work
This work provides a formal graphical criterion for understanding the limits of CRL. By moving from atomic to subset interventions, it aligns CRL with real-world data constraints. Future research will focus on scaling these theoretical insights to non-linear, high-dimensional data like video and exploring "soft" interventions that shift distributions rather than resetting variables.
Summary updated