Detecting low-complexity unobserved causes

Dominik Janzing, Eleni Sgouritsa, Oliver Stegle, Jonas Peters, Bernhard Schölkopf

TL;DR

This paper introduces a novel method to distinguish between a "direct" causal link ($X \to Y$) and a relationship mediated or confounded by a hidden, "low-complexity" variable (like a binary switch). By analyzing the geometry of conditional distributions $P(Y|X)$ within the space of all possible distributions, the authors can detect the "fingerprint" left behind by unobserved bottlenecks, offering a powerful tool for fields like statistical genetics.

What Problem Does This Paper Address?

In causal inference, a core challenge is Reichenbach’s Common Cause Principle: if two variables $X$ and $Y$ are statistically dependent, either $X$ causes $Y$, $Y$ causes $X$, or a third unobserved variable $W$ causes both.

Standard methods often struggle to distinguish a direct cause from a "spurious" correlation caused by a hidden variable, especially when we only observe $X$ and $Y$. This is a massive headache in genetics. For example, if a specific genetic marker (SNP) correlates with a disease, is that SNP the actual cause, or is it just physically close to the real causal SNP on the chromosome (a phenomenon called linkage disequilibrium)? The authors aim to solve this by identifying when a "low-complexity" (low-range) unobserved variable $Z$ is acting as a bottleneck between $X$ and $Y$.

Methods

The authors introduce the concept of pure conditionals.

  1. The Simplex Geometry: Imagine the set of all possible probability distributions of $Y$ as a geometric shape called a simplex. Every conditional distribution $P(Y|x)$ for a specific value of $x$ is a point in this simplex.
  2. Purity Definition: A conditional $P(Y|X)$ is "pure" if the distributions $P(Y|x)$ cannot be represented as mixtures (convex combinations) of other distributions without leaving the simplex. Intuitively, "pure" distributions sit on the "edges" or "corners" of the distribution space.
  3. The Bottleneck Effect: If a hidden variable $Z$ with a small range (e.g., binary) exists between $X$ and $Y$, then $P(Y|X)$ must be a mixture of the distributions $P(Y|z)$. This forces the observed $P(Y|x)$ points into the interior of the simplex, making them "non-pure."
  4. Implementation: To test this on real data, the authors use Kernel Density Estimation (KDE) to estimate the densities and calculate a "purity ratio." They also use Reproducing Kernel Hilbert Spaces (RKHS) and Gram matrix rank estimation to guess the dimensionality (complexity) of the latent variable $Z$.

Datasets

The paper evaluates the method using two primary approaches:

  • Synthetic Data: They generated data using linear additive noise models where the noise was a mixture of Gaussians (creating multi-modal distributions). This allowed them to strictly control the causal structure ($X \to Y$ vs. $X \leftarrow Z \to Y$).
  • Semi-empirical Genetic Data: They used real genomic data from a 250K SNP chip of the plant Arabidopsis thaliana (1,200 samples). They kept the real-world correlation structure of the SNPs but simulated the phenotype ($Y$) to create ground-truth causal and non-causal scenarios.

Key Insights and Ideas

  • The Geometry of Causality: The most striking idea is that the "directness" of a causal link is encoded in the extremal nature of its conditional distributions. Direct mechanisms tend to produce "extreme" distributions, while hidden bottlenecks "blur" these into the interior of the distribution space.
  • Invariance: Purity is a robust property; it remains invariant under any continuous one-to-one transformation of the effect variable $Y$.
  • Latent Reconstruction: Remarkably, the authors prove (Theorem 3) that if you assume a hidden binary cause exists and the underlying mechanism is pure, you can actually reconstruct the unobserved distributions $P(Y|Z=0)$ and $P(Y|Z=1)$ just from the observed $P(Y|X)$.
  • Complexity as a Filter: The method specifically targets "low-complexity" latents. It assumes that nature is more likely to have simple hidden confounders (like a single nearby SNP) than highly complex ones.

Key Results

  • Purity as a Discriminator: In synthetic tests, the purity ratio for direct causal links was consistently near zero, while confounded links yielded significantly higher values, successfully separating the two cases.
  • Outperforming Correlation: In the Arabidopsis simulation, the purity-based method significantly outperformed standard correlation analysis. When identifying causal SNPs, the purity method achieved a much higher Area Under the ROC Curve (AUC), especially in "noisy" scenarios where non-causal SNPs were highly correlated with the phenotype.
  • Robustness to Coupling: While both methods fail when variables are perfectly coupled (deterministic), the purity method maintains accuracy much longer as "corruption" or noise increases compared to naive correlation.

Limitations and Future Work

  • Sample Size Sensitivity: Because the method relies on density estimation (KDE), it requires a relatively large number of samples to accurately resolve the "tails" of the distributions, which are crucial for determining purity.
  • High-Complexity Latents: If the unobserved cause $Z$ is just as complex as $X$ (e.g., $Z$ is continuous and high-dimensional), the "bottleneck" signature disappears, and the method cannot distinguish it from a direct link.
  • Computational Scaling: The authors note that more efficient estimators for purity need to be developed for very high-dimensional or large-scale datasets.
  • Assumption of Purity: The method rests on the hypothesis that "natural" direct causal mechanisms tend to be pure. While they provide theoretical support (e.g., for additive noise models), this remains a foundational assumption that may not hold in all domains.

Summary updated

Abstract

We describe a method that infers whether statistical dependences between two observed variables X and Y are due to a "direct" causal link or only due to a connecting causal path that contains an unobserved variable of low complexity, e.g., a binary variable. This problem is motivated by statistical genetics. Given a genetic marker that is correlated with a phenotype of interest, we want to detect whether this marker is causal or it only correlates with a causal one. Our method is based on the analysis of the location of the conditional distributions P(Y|x) in the simplex of all distributions of Y. We report encouraging results on semi-empirical data.
BibTeX
@inproceedings{janzing2011detecting,
  author = {Dominik Janzing and Eleni Sgouritsa and Oliver Stegle and Jonas Peters and Bernhard Schölkopf},
  title = {{Detecting low-complexity unobserved causes}},
  booktitle = {Conference on Uncertainty in Artificial Intelligence},
  year = {2011},
  url = {https://openreview.net/forum?id=_dKaScpylF2},
}

Loading related content...

Figure