Detecting low-complexity unobserved causes

Dominik Janzing, Eleni Sgouritsa, Oliver Stegle, Jonas Peters, Bernhard Schölkopf

Introduction

In causal inference, a fundamental challenge is Reichenbach’s Common Cause Principle: if two variables $X$ and $Y$ are statistically dependent, either $X$ causes $Y$, $Y$ causes $X$, or a third unobserved variable $W$ causes both. Distinguishing a direct causal link from a relationship mediated or confounded by a hidden variable is a major hurdle in fields like statistical genetics. For instance, determining if a specific genetic marker (SNP) is the actual cause of a disease or merely in linkage disequilibrium with the true cause is critical for downstream research.

This paper introduces a novel method to detect the "fingerprint" of unobserved bottlenecks by analyzing the geometry of conditional distributions $P(Y|X)$. By focusing on "low-complexity" (low-range) hidden variables, the authors provide a powerful tool for identifying direct mechanisms.

The Geometry of Causality

The core innovation lies in viewing the set of all possible probability distributions of $Y$ as a geometric simplex. Every conditional distribution $P(Y|x)$ for a specific value of $x$ is represented as a point within this space.

The geometry of the bottleneck effect: observed conditional distributions $P(Y|x)$ are constrained to the interior of the simplex as mixtures of latent distributions $P(Y|z)$.

A distribution is defined as "pure" if it sits on the "edges" or "corners" of the simplex, meaning it cannot be represented as a mixture (convex combination) of other distributions. When a hidden variable $Z$ with a small range (e.g., binary) acts as a bottleneck between $X$ and $Y$, the observed $P(Y|x)$ must be a mixture of the latent distributions $P(Y|z)$. This forces the observed points into the interior of the simplex, making them "non-pure."

Methodology

To implement this geometric intuition, the authors utilize several advanced statistical tools:

  1. Kernel Density Estimation (KDE): Used to estimate the densities and calculate a "purity ratio" for the observed distributions.
  2. Reproducing Kernel Hilbert Spaces (RKHS): Employed alongside Gram matrix rank estimation to estimate the dimensionality and complexity of the latent variable $Z$.
  3. Latent Reconstruction: The authors demonstrate that if a hidden binary cause is assumed and the underlying mechanism is pure, the unobserved distributions $P(Y|Z=0)$ and $P(Y|Z=1)$ can be reconstructed solely from the observed $P(Y|X)$.

Experimental Evaluation

The method was evaluated using both synthetic and semi-empirical data:

  • Synthetic Data: Generated using linear additive noise models with multi-modal noise (mixtures of Gaussians) to strictly control causal structures.
  • Genetic Data: Simulated phenotypes based on real genomic data from Arabidopsis thaliana (1,200 samples), preserving real-world correlation structures while providing ground-truth causal scenarios.

Key Results

The purity ratio proved to be a robust discriminator. In synthetic tests, direct causal links ($X \to Y$) consistently yielded purity ratios near zero, while confounded links ($X \leftarrow Z \to Y$) showed significantly higher values, effectively separating the two cases.

Empirical results showing (a) the separation of purity ratios between direct and confounded links, and (b-d) the reconstruction of latent distributions.

In the Arabidopsis simulations, the purity-based method significantly outperformed standard correlation analysis. It achieved a much higher Area Under the ROC Curve (AUC) for identifying causal SNPs, particularly in "noisy" scenarios where non-causal SNPs were highly correlated with the phenotype. Furthermore, the method maintained accuracy longer than naive correlation as noise or "corruption" in the data increased.

Discussion and Limitations

While powerful, the method has specific requirements:

  • Sample Size: Relying on KDE means the method requires a relatively large number of samples to accurately resolve the distribution tails necessary for determining purity.
  • Complexity Constraints: If the unobserved cause $Z$ is as complex as $X$ (e.g., high-dimensional and continuous), the bottleneck signature disappears, and the method cannot distinguish it from a direct link.
  • Assumption of Purity: The framework rests on the hypothesis that natural direct causal mechanisms tend to be pure. While theoretically supported for additive noise models, this remains a foundational assumption for the approach.

Summary updated

BibTeX
@inproceedings{janzing2011detecting,
  author = {Dominik Janzing and Eleni Sgouritsa and Oliver Stegle and Jonas Peters and Bernhard Schölkopf},
  title = {{Detecting low-complexity unobserved causes}},
  booktitle = {Conference on Uncertainty in Artificial Intelligence},
  year = {2011},
  url = {https://openreview.net/forum?id=_dKaScpylF2},
}
Figure