Causal Inference by using Invariant Prediction: Identification and Confidence Intervals

Jonas Peters, Peter Bühlmann, Nicolai Meinshausen

Abstract

What is the difference between a prediction that is made with a causal model and that with a non-causal model? Suppose that we intervene on the predictor variables or change the whole environment. The predictions from a causal model will in general work as well under interventions as for observational data. In contrast, predictions from a non-causal model can potentially be very wrong if we actively intervene on variables. Here, we propose to exploit this invariance of a prediction under a causal model for causal inference: given different experimental settings (e.g. various interventions) we collect all models that do show invariance in their predictive accuracy across settings and interventions. The causal model will be a member of this set of models with high probability. This approach yields valid confidence intervals for the causal relationships in quite general scenarios. We examine the example of structural equation models in more detail and provide sufficient assumptions under which the set of causal predictors becomes identifiable. We further investigate robustness properties of our approach under model misspecification and discuss possible extensions. The empirical properties are studied for various data sets, including large-scale gene perturbation experiments.

Introduction

In the field of machine learning, the foundational principle that "correlation does not imply causation" is often cited, yet most predictive models rely heavily on correlations. These models can be brittle; when an environment changes—due to policy shifts, laboratory interventions, or natural variation—correlation-based relationships often collapse. In their seminal paper, "Causal Inference using Invariant Prediction: Identification and Confidence Intervals," Jonas Peters, Peter Bühlmann, and Nicolai Meinshausen propose a shift in perspective: identifying causal relationships by searching for predictors that yield a stable model across multiple experimental settings.

The Invariance Principle

Standard predictive models are prone to failure under distribution shifts. If a model is trained to predict a target $Y$ using features $X$, it captures the specific joint distribution of that training data. If an intervention occurs, such as a gene knockout or a price change, the relationship between $X$ and $Y$ may vanish. However, a causal model should be robust to such changes. If $X$ truly causes $Y$, the mechanism $P(Y|X_{causal})$ should remain invariant regardless of interventions on other variables in the system.

The Invariant Causal Prediction (ICP) Algorithm

The authors operationalize this principle through the Invariant Causal Prediction (ICP) algorithm. The method requires data from at least two distinct environments (e.g., observational vs. interventional data).

Structural equation models across three environments. The "hammers" represent interventions on specific variables, creating different experimental settings.

ICP proceeds by searching over every possible subset of predictors $S$. For each subset, it tests the null hypothesis that the conditional distribution $P(Y|X_S)$ is identical across all environments. The set of identifiable causal predictors is then defined as the intersection of all subsets $S$ that were not rejected by the invariance test. Because the true causal parents are theoretically invariant, they will be among the accepted sets, and their intersection provides a conservative estimate with rigorous statistical guarantees.

Two implementations are proposed: Method I, which uses a Chow-test-like approach to assess regression coefficient stability, and Method II, a faster version that checks if residuals have consistent mean and variance across environments.

Empirical Validation

The authors validate ICP across synthetic data, socio-economic studies, and biological experiments. A notable application is the Yeast Gene-Perturbation dataset, which includes 160 observational samples and 1,479 samples with single-gene deletions.

Scatter plots of gene expression levels in yeast. The blue crosses represent observational data, while the orange dots represent interventional data across different environments.

In the yeast study, ICP identified eight significant causal gene pairs. When validated against ground-truth knockout data, six out of the eight predictions were correct, significantly outperforming standard correlation-based methods and other causal discovery tools like IDA. This demonstrates the method's ability to provide a Family-Wise Error Rate (FWER) guarantee, a critical feature for researchers prioritizing precision in target identification.

Key Insights and Limitations

  • Unknown Intervention Targets: ICP does not require knowledge of which specific variables were intervened upon, only that the data originates from different "blocks" or environments.
  • Safe Misspecification: In the presence of hidden confounders, ICP tends to return an empty set rather than making false causal claims, prioritizing reliability.
  • Scalability and Linearity: The current search over all subsets is computationally intensive ($2^p$), and the primary focus is on linear models. Extending these statistical tests to high-dimensional, non-linear settings remains an active area of research.

Conclusion

Invariant Causal Prediction provides a powerful framework for causal inference by leveraging the stability of causal mechanisms. By focusing on invariance across environments, it offers a path toward more robust and reliable predictive modeling in complex, changing systems.

Summary updated

Abstract

Summary What is the difference between a prediction that is made with a causal model and that with a non-causal model? Suppose that we intervene on the predictor variables or change the whole environment. The predictions from a causal model will in general work as well under interventions as for observational data. In contrast, predictions from a non-causal model can potentially be very wrong if we actively intervene on variables. Here, we propose to exploit this invariance of a prediction under a causal model for causal inference: given different experimental settings (e.g. various interventions) we collect all models that do show invariance in their predictive accuracy across settings and interventions. The causal model will be a member of this set of models with high probability. This approach yields valid confidence intervals for the causal relationships in quite general scenarios. We examine the example of structural equation models in more detail and provide sufficient assumptions under which the set of causal predictors becomes identifiable. We further investigate robustness properties of our approach under model misspecification and discuss possible extensions. The empirical properties are studied for various data sets, including large-scale gene perturbation experiments.
BibTeX
@article{peters2016causal,
  author = {Jonas Peters and Peter Bühlmann and Nicolai Meinshausen},
  title = {{Causal Inference by using Invariant Prediction: Identification and Confidence Intervals}},
  journal = {Journal of the Royal Statistical Society Series B (Statistical Methodology)},
  year = {2016},
  doi = {10.1111/rssb.12167},
}
Figure