Breaking Observational Symmetry in Bivariate Causal Discovery

30 concepts24 papers

A fundamental rule of statistics is that correlation does not imply causation. If we observe two variables, $X$ and $Y$, that move together, standard statistical tools cannot tell us whether $X$ causes $Y$ ($X \to Y$) or $Y$ causes $X$ ($Y \to X$).

In larger systems with three or more variables, we can often orient causal arrows by looking for conditional independencies. As shown by Dan Geiger et al. in Learning Causal Trees from Dependence Information, the presence of a third variable allows us to identify unique patterns—such as v-structures—that reveal the underlying graph.

But when we are down to just two variables, we face a wall of observational symmetry. The joint probability distribution factors perfectly in both directions:

$$P(X, Y) = P(X)P(Y|X) = P(Y)P(X|Y)$$

Without a third variable or a temporal clock, how can we determine the direction of influence? The answer lies in finding subtle structural footprints left behind by the true causal mechanism.

The Independence of Cause and Mechanism

The foundational philosophy for breaking this symmetry is the principle of Independence of Cause and Mechanism (ICM). As discussed by Rong Jiao et al. in Bivariate Causal Discovery and its Applications to Gene Expression and Imaging Data Analysis, this principle posits that the source distribution (the cause, $P(X)$) and the physical process that transforms the cause into the effect (the mechanism, $P(Y|X)$) are chosen by nature independently. They contain no information about one another.

If $X \to Y$ is the true direction, changing the distribution of the cause $P(X)$ will leave the conditional mechanism $P(Y|X)$ completely unchanged. However, if we force ourselves to look at the anti-causal direction ($Y \to X$), the marginal distribution $P(Y)$ and the conditional distribution $P(X|Y)$ must intricately align to cancel each other out.

To exploit this, Xiaohai Sun et al. proposed evaluating the smoothness of these conditional distributions in Causal Inference by Choosing Graphs with Most Plausible Markov Kernels. They showed that the true causal direction yields a highly plausible, independent mechanism, whereas the wrong direction requires a highly complex, fine-tuned distribution.

Structural Asymmetry via Additive Noise

To turn this philosophical principle into an algorithm, researchers restrict the functional form of the mechanism. The most prominent framework is the Additive Noise Model (ANM). Suppose the true relationship is:

$$Y = f(X) + \epsilon, \quad \epsilon \perp X$$

where the noise $\epsilon$ is strictly independent of the cause $X$.

The core insight of ANMs is a profound mathematical asymmetry: if $f$ is non-linear, it is generally impossible to invert the equation such that $X = g(Y) + \eta$ with $\eta$ independent of $Y$.

To see why, imagine a non-linear function $f(X)$. The independent noise $\epsilon$ creates a uniform "band" of uncertainty along the y-axis across all values of $X$. If you project this distribution back onto the x-axis to predict $X$ from $Y$, the width of the noise will vary depending on the slope of $f(X)$. Where the function is steep, the spread of $X$ given $Y$ will be narrow; where it is flat, the spread will be wide. Consequently, the residual error in the reverse direction inherently depends on $Y$.

True Direction (X -> Y):          Anti-Causal Direction (Y -> X):
      Y |   . . . . .                   Y |      .  .
        |  . . . . .                      |    . . . .
        | . . . . .                       |   . . . . .
        +------------                     +------------
              X                                 X
  (Noise variance is uniform)       (Noise variance depends on Y)

This asymmetry provides a clear signature for causal direction. Jonas Peters et al. extended this logic to discrete spaces in Causal Inference on Discrete Data Using Additive Noise Models, proving that the independent noise signature holds for integers and categories. For scenarios where sensors introduce distortions, Kun Zhang et al. introduced the post-nonlinear model in Nonlinear acyclic causal models, demonstrating that the directional asymmetry persists even after non-linear transformations of the observed variables. Even when noise is not strictly additive, probabilistic latent variable models developed by Joris M. Mooij et al. in Probabilistic latent variable models for distinguishing between cause and effect can successfully isolate these directional signatures.

Algorithmic Complexity and Data Compression

Another way to operationalize the independence of cause and mechanism is through algorithmic information theory and the Minimum Description Length (MDL) principle.

If $P(X)$ and $P(Y|X)$ are independent, the shortest computer program to describe their joint distribution is simply the sum of the shortest programs describing each piece separately. Conversely, because the anti-causal factorization $P(Y)P(X|Y)$ requires fine-tuning, its description length will be longer; the mechanism $P(X|Y)$ must implicitly encode information about the distribution of the effect $P(Y)$ to make the math work.

As a result, the true causal direction allows for more succinct data compression. In Telling Cause from Effect Using MDL-Based Local and Global Regression, Alexander Marx et al. introduced the SLOPE algorithm, which fits regressions in both directions and selects the one that minimizes the total description length.

This compression perspective scales remarkably well with modern deep learning. Quang-Duy Tran et al. utilized neural networks and variational Bayesian inference in Identifying Causal Direction via Variational Bayesian Compression to find the most compact representation of the data.

The same principle can be framed using standard information entropy. As shown by Alex Dimakis et al. in Entropic Causal Inference, forcing an entropic model in the wrong direction artificially inflates the entropy of the inferred exogenous noise variable, providing another clear mathematical indicator of the true causal arrow.

Geometric and Statistical Footprints

The asymmetry between cause and effect also leaves simpler geometric footprints in regression errors. Patrick Blöbaum et al. demonstrated a remarkably straightforward approach in Cause-Effect Inference by Comparing Regression Errors: fit a standard regression in both directions and pick the one with the smaller mean squared error. Because of how noise propagates through non-linear functions, the true causal direction is inherently "easier" for a model to fit.

True Direction: Low Error         Anti-Causal Direction: High Error
      Y |   /                           Y |   |
        |  /                              |   |  __--
        | /                               |  |/
        +------------                     +------------
              X                                 X

When data exhibits heteroscedasticity—meaning the noise variance changes depending on the input value—traditional mean-based regressions fail to capture the full picture. To solve this, Natasa Tagasovska et al. developed a quantile-based approach in Distinguishing Cause from Effect Using Quantiles: Bivariate Quantile Causal Discovery. By analyzing how well different "slices" (quantiles) of the data can be compressed, they showed that the true causal direction can be uncovered even when the noise structure is highly variable.

Scaling to Complex Environments

As the field has matured, researchers have moved beyond rigid functional constraints to handle highly complex, real-world distributions. Rather than assuming specific noise structures, David Lopez-Paz et al. reframed causal discovery as a supervised learning problem in Towards a Learning Theory of Cause-Effect Inference. By using kernel mean embeddings, they transformed entire probability distributions into feature vectors, training a classifier to recognize the subtle geometric differences between cause and effect.

Similarly, Dino Sejdinovic et al. leveraged Reproducing Kernel Hilbert Spaces (RKHS) in Causal Inference via Kernel Deviance Measures to nonparametrically quantify the structural variability of conditional distributions, bypassing the need for explicit functional assumptions.

To handle complex functional relationships, Pengzhou Wu et al. combined nonlinear Independent Component Analysis (ICA) with neural network ensembles in Causal Mosaic: Cause-Effect Inference via Nonlinear ICA and Ensemble Method. Even when unobserved variables sit between a cause and its effect, modern generative approaches like the conditional diffusion framework proposed by Dominik Meier et al. in When Additive Noise Meets Unobserved Mediators: Bivariate Denoising Diffusion for Causal Discovery can successfully isolate and test the independence of the underlying causal mechanisms.

By shifting the focus from simple statistical correlations to structural, algorithmic, and geometric asymmetries, bivariate causal discovery proves that the direction of the causal arrow is often hidden right in the data itself.

Key Foundational Work

Causal Inference by Choosing Graphs with Most Plausible Markov Kernels (2006) — contributed 1 concept elements to this direction.
Learning Causal Trees from Dependence Information (1990) — contributed 1 concept elements to this direction.
History & citationVersion 2 · 2026-10-04

Current · Permanent link

  1. Version 2 (latest) · 2026-10-04 · 25 papers
  2. Version 1 · 2026-10-04 · 43 papers
Cite this direction

Latest version (a living page):

@misc{lacuna_bivariate_causal_discovery_c8572edb,
  title = {bivariate causal discovery},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/bivariate-causal-discovery/rdr_c8572edbc08d41768e77ffc8d8a97768},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 2, frozen text):

@misc{lacuna_bivariate_causal_discovery_c8572edb_v2,
  title = {bivariate causal discovery},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/bivariate-causal-discovery/rdr_c8572edbc08d41768e77ffc8d8a97768/v/2},
  note = {Version 2, frozen 2026-10-04. Accessed 2026-11-20},
}

See something wrong? Report a mistake