Causal Discovery with Additive Noise Models

This direction is merged into another direction: non-Gaussian causal identifiability. Its last version stays readable and citable here.

Version 1 of 121 papers25 conceptsfrozen 2026-10-04

In classical statistics, the mantra "correlation does not imply causation" is often treated as a mathematical dead end. If you observe two variables, $X$ and $Y$, that are statistically dependent, the joint distribution $P(X, Y)$ is symmetric. Mathematically, $P(X)P(Y|X)$ is identical to $P(Y)P(X|Y)$. Without a randomized controlled trial, it was long believed impossible to determine if $X$ causes $Y$ or vice versa.

However, a body of research led by Bernhard Schölkopf, Jonas Peters, and Dominik Janzing has demonstrated that this symmetry is an artifact of overly simple models. By looking at the "noise" or residuals of a relationship, we can find a physical footprint that reveals the direction of the causal arrow.

The Intuition of Asymmetry

The core idea rests on the Additive Noise Model (ANM). We assume that if $X$ causes $Y$, the relationship can be described as: $$Y = f(X) + N$$ where $f$ is some function and $N$ is a noise term that is statistically independent of the cause $X$.

The breakthrough, described in Nonlinear causal discovery with additive noise models by Patrik O. Hoyer et al., is that this independence usually only holds in the true causal direction. If you try to fit the model in reverse—assuming $Y$ causes $X$—the resulting noise term will almost always show a statistical dependency on $Y$.

Imagine a scatter plot where $Y = X^3 + N$. If you regress $Y$ on $X$, the residuals (the "vertical" distance from the curve) look like random, uniform noise across all values of $X$. But if you try to regress $X$ on $Y$, the "horizontal" residuals will be squeezed or stretched depending on the value of $Y$. This statistical asymmetry allows us to distinguish cause from effect.

The Blessing of Nonlinearity

In linear Gaussian models, this trick doesn't work; the symmetry remains perfectly preserved. However, Peter Spirtes and others have noted that the real world is rarely linear or Gaussian.

As Jonas Peters et al. argue in Identifiability of Causal Graphs using Functional Models, nonlinearity and non-Gaussian noise are actually a "blessing." They provide the structural information needed to identify the causal direction. This moves us beyond "Markov equivalence classes"—groups of graphs that look the same to traditional tests—to uniquely identifiable causal structures.

Testing the Theory in the Wild

To move these theories from math to reality, Joris M. Mooij and colleagues introduced the CauseEffectPairs benchmark in Distinguishing cause from effect using observational data. This dataset contains real-world pairs where the causal direction is known from physical laws (e.g., altitude causes temperature, not the other way around).

The ANM approach successfully recovers these directions by checking which orientation yields independent residuals. As Bernhard Schölkopf explains in Causality for Machine Learning, this is a fundamental shift for AI. While standard machine learning excels at "anticausal" tasks (predicting a cause from an effect, like identifying an object from an image), true intelligence requires "causal" modeling to predict how a system will react to interventions.

Modern Frontiers: Missing Data and Deep Learning

Recent work has extended the ANM framework to handle the messiness of real-world data.

Despite these advances, challenges remain. Rhys Peter Matthew Howard and Lars Kunze have pointed out a "reality gap" where methods that work on synthetic data struggle with the non-stationary behavior of humans, such as drivers on a road.

To help bridge this gap, tools like the causal-learn Python library by Yujia Zheng et al. are making these sophisticated functional models accessible to the broader machine learning community, moving causal discovery from a theoretical niche into a practical tool for scientific inquiry.

Papers in this version

History & citationVersion 1 · 2026-10-04

Merged into another direction · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_causal_discovery_with_additive_noise_mod_8d88a061,
  title = {Causal Discovery with Additive Noise Models},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/causal-discovery-with-additive-noise-models/rdr_8d88a061bd094306b210a039a00e4a02},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_causal_discovery_with_additive_noise_mod_8d88a061_v1,
  title = {Causal Discovery with Additive Noise Models},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/causal-discovery-with-additive-noise-models/rdr_8d88a061bd094306b210a039a00e4a02/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}