Causal Discovery Beyond the Additive Noise Assumption

124 concepts108 papers1 proposal

Determining whether $X$ causes $Y$ or $Y$ causes $X$ from purely observational data is one of the fundamental challenges of statistics. For decades, standard linear models were stymied by a fundamental symmetry: if $X$ and $Y$ are jointly Gaussian, the statistical footprint of $X \to Y$ is mathematically indistinguishable from $Y \to X$.

A breakthrough came when researchers realized that nonlinearity breaks this symmetry. As demonstrated by Hoyer et al. (2008), if the relationship between cause and effect is nonlinear, we can uniquely identify the true causal direction. However, this "blessing of nonlinearity" originally relied on a strict structural constraint: the Additive Noise Model (ANM).

An ANM assumes that the effect is a deterministic function of the cause plus an independent background disturbance:

$$Y = f(X) + \epsilon, \quad \epsilon \perp \!\!\! \perp X$$

While mathematically convenient, real-world systems rarely confine their randomness to a neat additive term at the end of an equation. Noise can be multiplicative, heteroscedastic, or deeply intertwined with the causal mechanism itself. Over-reliance on the additive noise assumption causes traditional discovery methods to fail when confronted with the messy, non-additive realities of empirical data.


Why Additive Noise Models Work (and Where They Break)

To understand why the additive noise assumption is both powerful and fragile, consider the geometry of a causal relationship.

If the true data-generating process follows an ANM ($X \to Y$), regressing $Y$ on $X$ yields a residual $\hat{\epsilon} = Y - f(X)$ that is completely independent of $X$. If we attempt to fit the model in the reverse direction ($X = g(Y) + \eta$), the nonlinearity of $f$ distorts the distribution of the noise. The resulting reverse residuals $\hat{\eta}$ will almost always display a statistical dependence on $Y$. This asymmetry provides a clear directional signal.

True Direction (X -> Y):      Y = f(X) + ε  ==>  Residuals are independent of X
Incorrect Direction (Y -> X):  X = g(Y) + η  ==>  Residuals remain dependent on Y

But what happens if the true mechanism is non-additive, such as a multiplicative interaction ($Y = f(X) \cdot \epsilon$)?

If we force an additive model onto this system, the variance of our residuals will scale with the magnitude of $X$. Because the noise variance is modulated by the cause, any additive residual we calculate will inherently depend on $X$. As noted by Mooij et al. (2010) and Yamada et al. (2014), this violation destroys the independence guarantee in both directions. The causal discovery algorithm will either conclude that no causal link exists or, worse, misidentify the direction entirely.


Generalizing the Functional Form: Post-Nonlinear Models

The first major step away from strict additivity was the introduction of the Post-Nonlinear (PNL) causal model by Zhang et al. (2010) and Zhang et al. (2009).

The PNL framework introduces a second nonlinear function, $g$, that acts after the noise has been additively mixed:

$$Y = g(f(X) + \epsilon)$$

This formulation elegantly captures real-world scenarios like sensor distortions, where a physical process might be additive, but the measurement apparatus introduces a non-linear warp. By framing causal discovery as a constrained nonlinear Independent Component Analysis (ICA) problem, the authors proved that the causal direction remains identifiable in almost all cases, significantly expanding the types of non-additive systems researchers could reliably untangle.


Information-Theoretic and Kernel-Based Alternatives

Rather than trying to guess the exact mathematical shape of how noise interacts with a cause, another branch of research bypasses functional equations entirely by looking at the conditional distributions directly.

Sejdinovic et al. (2018) introduced Kernel Conditional Deviance for Causal Inference (KCDC). Instead of performing a regression and subtracting residuals, KCDC leverages Reproducing Kernel Hilbert Spaces (RKHS) to measure the "structural variability" of conditional distributions. The intuition is that the mechanism generating an effect from a cause is inherently more stable and less complex than the mathematical fiction required to generate a cause from its effect.

Similarly, Sgouritsa et al. (2015) developed CURE (Causal inference with Unsupervised inverse REgression). CURE exploits an information-theoretic asymmetry: the marginal distribution of a true cause $P(X)$ contains no information about the conditional mechanism $P(Y|X)$. However, because the effect is a product of the cause, the marginal distribution of the effect $P(Y)$ will contain structural "footprints" of $P(X)$. By quantifying this algorithmic information asymmetry, these methods identify causal direction without needing to assume that noise is additive or even homoscedastic.


Continuous Dynamics and Generative Modeling

With the rise of deep generative modeling, researchers have developed noise-agnostic frameworks capable of handling arbitrary, inseparable interactions between variables and noise in high dimensions.

Causal Velocity Models

Xi et al. (2025) reframe bivariate causal discovery as a dynamical system problem through Causal Velocity Models (CVMs). Instead of viewing the cause as a static input to a noisy function, CVMs treat the cause variable as "time" and model how the probability density of the effect flows using the continuity equation. By linking the causal mechanism directly to the data's score function (the gradient of the log density), CVMs can distinguish cause from effect in highly complex, heteroscedastic scenarios where traditional residual-based methods fail.

Normalizing Flows and Diffusion

Other approaches leverage invertible neural networks to isolate complex noise structures. Khemakhem (2021) introduced Causal Autoregressive Flows (CAREFL), which use restricted normalizing flows to model causal mechanisms where the noise variance is explicitly modulated by the causal variable.

For interventional and counterfactual queries, Chao et al. (2024) demonstrated that diffusion models can be treated as deterministic causal mechanisms. Because diffusion processes smoothly transition data to a latent noise space, they can capture non-additive, inseparable interactions between observed variables and unobserved disturbances, allowing for accurate counterfactual reasoning without the crutch of the additive noise assumption.

Key Foundational Work

Nonlinear causal discovery with additive noise models (2008) — contributed 1 concept elements to this direction.
History & citationVersion 2 · 2026-10-04

Current · Permanent link

  1. Version 2 (latest) · 2026-10-04 · 105 papers
  2. Version 1 · 2026-10-04 · 146 papers
Cite this direction

Latest version (a living page):

@misc{lacuna_additive_noise_assumption_4a9fc3d9,
  title = {additive noise assumption},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/additive-noise-assumption/rdr_4a9fc3d9b52c4f3492691347196d350f},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 2, frozen text):

@misc{lacuna_additive_noise_assumption_4a9fc3d9_v2,
  title = {additive noise assumption},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/additive-noise-assumption/rdr_4a9fc3d9b52c4f3492691347196d350f/v/2},
  note = {Version 2, frozen 2026-10-04. Accessed 2026-11-20},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake