Statistical test for consistent estimation of causal effects in linear non-Gaussian models

Doris Entner, Patrik O. Hoyer, Peter SpirtesAISTATS 2012 · OpenReview

Abstract

In many fields of science, researchers are faced with the problem of estimating causal effects from non-experimental data. A key issue is to avoid inconsistent estimators due to confounding by measured or unmeasured variables. This paper introduces a statistical test for linear models that determines if a specific set of covariates is sufficient for a consistent causal estimate, even when the underlying graph is unknown. By leveraging the unique properties of non-Gaussian noise, the authors provide a way to "sanity check" causal claims using only partial knowledge of temporal ordering.

Introduction

In fields like epidemiology or economics, researchers often want to know the causal effect of a treatment ($x$) on an outcome ($y$). Without a randomized controlled trial, they must "adjust for" other variables (covariates) to block confounding paths. The gold standard for choosing these covariates is the back-door criterion, but this requires knowing the full Directed Acyclic Graph (DAG) of the system. If the graph is unknown, researchers often resort to "adjusting for everything," which can actually introduce more bias (e.g., by conditioning on a "collider"). While some algorithms can learn the whole graph from data, they are often computationally expensive and struggle when some variables are unmeasured (latent). This paper seeks a middle ground: a simple test to see if a specific adjustment set works, without needing the full graph.

Theoretical Framework and Methodology

The authors' approach relies on a key mathematical insight: in linear models with non-Gaussian disturbance terms (noise), the "wrong" adjustment set leaves a statistical "fingerprint" in the residuals. The theoretical backbone is the Darmois-Skitovitch Theorem, which states that if linear combinations of independent random variables are themselves independent, then the variables must be Gaussian. In the non-Gaussian world, independence is a much stricter and more informative condition than mere lack of correlation.

Structural models and equations for causal systems with observed and latent variables.

The Independence Test

To test if a set $Z$ is a good adjustment set, the paper proposes a three-step algorithm:

  1. Regress $x$ on $Z$ to get the residual $r_x$.
  2. Regress $y$ on $x$ and $Z$ to get the residual $r_y$.
  3. Test for statistical independence between $r_x$ and $r_y$.

If $Z$ is a valid adjustment set, $r_x$ and $r_y$ will be independent. If $Z$ is invalid (leaving confounding or including colliders), the residuals will remain dependent, even if they are linearly uncorrelated. Since testing every possible subset of covariates is computationally expensive ($2^{|W|}$ subsets), the authors provide Forward Selection and Backward Elimination search heuristics to find a valid set $Z$ in quadratic time.

Experimental Evaluation

Simulated Data

The authors generated 5,000 random linear models with varying complexities (5–10 observed covariates and 3–5 hidden variables). They tested sample sizes ranging from 100 to 100,000 to evaluate how the test scales. In these simulations, higher p-values for the independence test (meaning the residuals were more likely to be independent) correlated strongly with lower errors in the estimated causal effect.

Relationship between the p-value of the independence test and the error in the causal effect estimate (10,000 samples).

Real-World Application: Car Mileage Data

A real-world dataset containing 82 observations of vehicle specs (weight, horsepower, top speed, cab space) and fuel efficiency (MPG) was used to demonstrate the test. The test identified that the effect of horsepower on MPG was consistent when adjusting for nothing or just "cab space," yielding a stable estimate of approximately -0.66. It correctly rejected sets that would have led to intuitively wrong signs, such as weight having a positive effect on top speed.

Key Insights and Limitations

  • Non-Gaussianity as a Feature: Usually, non-normal data is seen as a nuisance. Here, it is the "secret sauce" that allows the test to work. Gaussian distributions are too symmetric to reveal causal directionality through residuals alone.
  • Minimal Prior Knowledge: You don't need the full DAG. You only need to know that your covariates ($W$) aren't caused by the treatment ($x$) and that the outcome ($y$) doesn't cause the covariates. This "temporal ordering" is often much easier for experts to provide than a full structural model.
  • Assumptions: The method assumes the relationships between variables are linear. If the underlying noise is Gaussian, the test has no power, as it cannot distinguish between consistent and inconsistent sets. Furthermore, like many non-parametric independence tests, this method requires a relatively large number of samples (typically $N \ge 1000$) to be reliable.

Summary updated

Abstract

In many fields of science researchers are faced with the problem of estimating causal effects from non-experimental data. A key issue is to avoid inconsistent estimators due to confounding by measu...
BibTeX
@inproceedings{entner2012statistical,
  author = {Doris Entner and Patrik O. Hoyer and Peter Spirtes},
  title = {{Statistical test for consistent estimation of causal effects in linear non-Gaussian models}},
  booktitle = {International Conference on Artificial Intelligence and Statistics},
  year = {2012},
  url = {https://openreview.net/forum?id=bBn8LQ0VUc7},
}
Figure