Computational causal discovery: Advantages and assumptions

Kun ZhangTHEORIA 2022 · DOI

TL;DR

This paper bridges the gap between philosophical theories of "interventionist" causality and modern machine learning algorithms. It explains how we can uniquely identify causal direction from purely observational data—even between just two variables—by exploiting the "footprints" left by natural variations in non-Gaussian noise.

What Problem Does This Paper Address?

In science, the gold standard for proving that $X$ causes $Y$ is the randomized controlled trial (RCT). By intervening on $X$ and observing changes in $Y$, we confirm a causal link. However, in fields like epidemiology, economics, or climate science, experiments are often impossible, unethical, or prohibitively expensive.

Researchers are then left with "non-experimental" (observational) data. The challenge is that "correlation is not causation." Standard statistical methods often produce a set of possible causal structures (Markov Equivalence Classes) but cannot tell you which way the arrow points—especially when you only have two variables. This paper explores how emerging computational methods move beyond these limits to find the "true" causal direction by looking at the underlying functional mechanisms of the data.

Methods

The author, Kun Zhang, focuses on Independent Noise-Based Approaches, contrasting them with older conditional independence methods (like the PC algorithm).

Key techniques discussed include:

  • Functional Causal Models (FCMs): These model an effect as a function of its cause and an independent noise term (e.g., $Y = f(X) + \text{noise}$).
  • LiNGAM (Linear Non-Gaussian Acyclic Model): This is a cornerstone of the paper. It demonstrates that if a relationship is linear and the noise is non-Gaussian (not a standard bell curve), the causal direction becomes "identifiable." In this framework, the noise will only be independent of the predictor in the correct causal direction.
  • Overcomplete ICA (Independent Component Analysis): Used to handle complex scenarios where hidden "confounders" (unmeasured variables that influence both $X$ and $Y$) might be present.

The paper essentially argues that these algorithms look for "natural interventions"—random fluctuations in the data that act like mini-experiments performed by nature.

Datasets

The paper is a theoretical and methodological review; it does not introduce new empirical datasets or benchmarks. Instead, it uses illustrative examples to prove its points, such as:

  • The relationship between atmospheric pressure and barometer readings.
  • A hypothetical model involving lifestyle, mortality risk, food consumption, and physical activity to demonstrate how hidden variables can mislead or be accounted for.

Key Insights and Ideas

  • The Power of Non-Gaussianity: One of the most striking "aha moments" is that in a perfectly linear, Gaussian world, causal direction is mathematically invisible. However, the real world is rarely perfectly Gaussian. By exploiting the "skewness" or "flatness" of real-world noise, we can break the symmetry and see which variable is the driver.
  • Asymmetry as a Compass: Causal discovery is essentially a search for asymmetry. If we assume a specific model class (like a linear model), we find that the "independent noise" property only holds in the direction of $Cause \rightarrow Effect$.
  • Faithfulness to the Truth: The paper emphasizes that causal discovery is fundamentally different from standard Machine Learning (ML). While standard ML cares about prediction (finding any pattern that works), causal discovery cares about truth (finding the actual mechanism).
  • Handling the Unseen: Even when we can't measure everything (confounders), we can still infer direction if we assume "faithfulness"—the idea that causal effects don't perfectly cancel each other out by some freak statistical accident.

Key Results

  • Unique Identifiability: The paper highlights that while older methods (PC/FCI) might tell you $X$ and $Y$ are related, newer noise-based methods (LiNGAM, ANM, PNL) can uniquely determine that $X \rightarrow Y$.
  • Theoretical Justification: It successfully connects James Woodward’s philosophical "interventionist" account of causation to the technical assumptions required by ML algorithms, providing a rigorous foundation for why these algorithms work.
  • Robustness to Confounders: It provides a technical account (Assumptions AP1 and AP2) showing that causal direction can be recovered even in the presence of hidden variables, provided those variables have non-zero effects on their descendants.

Limitations and Future Work

  • Untestable Assumptions: Some critical assumptions, such as the "non-zero total causal effect" of hidden variables, are not generally testable. We have to assume they hold to trust the output.
  • Model Sensitivity: The success of these methods depends heavily on choosing the right "functional class." If you assume a linear relationship but the true mechanism is highly non-linear, the algorithm may produce incorrect results.
  • Computational Complexity: While the paper discusses the theory, applying these methods to high-dimensional systems with thousands of variables remains a significant engineering challenge.
  • Bridging the Gap: The author notes that researchers must continue to bridge the gap between "statistical properties" and "causal interpretations" to ensure that discovery results are not just mathematical artifacts but represent real-world truths.

Summary updated

Abstract

I would like to congratulate James Woodward for another landmark accomplishment, after publishing his Making things happen: A theory of causal explanation (Woodward, 2003). Making things happen gives an elegant interventionist theory for understanding explanation and causation. The new contribution (Woodward, 2022) relies on that theory and further makes a big step towards empirical inference of causal relations from non-experimental data. In this paper, I will focus on some of the emerging computational methods for finding causal relations from non-experimental data and attempt to complement Woodward's contribution with discussions on 1) how these methods are connected to the interventionist theory of causality, 2) how informative the output of the methods is, including whether they output directed causal graphs and how they deal with confounders (unmeasured common causes of two measured variables), and 3) the assumptions underlying the asymptotic correctness of the output of the methods about causal relations. Different causal discovery methods may rely on different aspects of the joint distribution of the data, and this discussion aims to provide a technical account of the assumptions.
BibTeX
@inproceedings{zhang2022computational,
  author = {Kun Zhang},
  title = {{Computational causal discovery: Advantages and assumptions}},
  booktitle = {THEORIA},
  year = {2022},
  doi = {10.1387/theoria.22904},
}
Figure