Imagine you are given a dataset containing two continuous variables, $X$ and $Y$, and you want to determine whether $X$ causes $Y$ ($X \rightarrow Y$) or $Y$ causes $X$ ($Y \rightarrow X$). Traditional causal discovery methods might look for conditional independence patterns or fit complex non-linear equations. However, a simpler statistical property often gives away the answer: the variance of the variables.
In an Additive Noise Model (ANM), an effect variable is generated by a function of its causes plus an independent noise term. Consider the simplest linear case:
$$X = \epsilon_X$$ $$Y = \alpha X + \epsilon_Y$$
If the exogenous noise terms $\epsilon_X$ and $\epsilon_Y$ have equal or comparable variances, the marginal variance of the effect variable $Y$ naturally inflates because it inherits the variance of its parent $X$ in addition to its own intrinsic noise:
$$\text{Var}(Y) = \alpha^2 \text{Var}(X) + \text{Var}(\epsilon_Y)$$
As long as the causal coefficient $\alpha$ is not close to zero, $\text{Var}(Y)$ will systematically exceed $\text{Var}(X)$. Along a directed acyclic graph (DAG), this creates a statistical gradient where marginal variance accumulates downstream from causes to effects.
This fundamental insight has sparked a major wave of research. It acts both as a dangerous pitfall for benchmarking algorithms and as a powerful mathematical shortcut for provably identifying causal structures.
The Benchmark Trap: Varsortability and $R^2$-Sortability
For years, researchers evaluated new causal discovery algorithms on synthetic datasets generated by randomly sampling edge weights and adding independent noise. However, Alexander G. Reisach et al. exposed a critical flaw in this paradigm: many state-of-the-art continuous optimization methods were not learning deep structural relationships. Instead, they were inadvertently exploiting the downstream accumulation of variance.
The authors coined the term varsortability to describe the degree to which the true causal order aligns with the sorted order of marginal variances. In standard synthetic benchmarks, varsortability is exceptionally high. A trivial baseline that simply sorts variables by their marginal variance and runs a sparse regression can match or outperform highly complex models.
A natural countermeasure is to standardize the data to unit variance before running an algorithm, which seemingly erases the marginal variance gradient. However, the authors followed up in A Scale-Invariant Sorting Criterion to Find a Causal Order in Additive Noise Models to show that a scale-invariant version of this artifact persists: $R^2$-sortability. Downstream variables are inherently more "explainable" by the rest of the dataset because they aggregate information from multiple ancestors. Consequently, sorting variables by their coefficient of determination ($R^2$) when regressed against all other variables still yields a highly accurate topological sort, even on standardized data.
This creates a massive generalization gap. As Konstantin Göbler et al. demonstrate, real-world industrial data does not exhibit these clean, artificial variance gradients. Algorithms that rely on varsortability shortcuts fail completely when deployed in practice. Similarly, Erich Kummerfeld et al. note that these hidden statistical artifacts compromise the integrity of benchmark performance metrics, making it difficult to assess how models handle genuine signal-to-noise ratios.
Turning a Bug into a Feature: Identifiability Guarantees
While varsortability is a dangerous confounder for benchmarking, the underlying mathematical relationship between variance and causal order can be formalized into powerful, provably correct algorithms. If we explicitly assume that all error terms have equal variance, the causal graph becomes fully identifiable from observational data.
Under the equal-variance assumption, a source node (a node with no parents) will always have the minimum conditional variance. The authors of A polynomial-time algorithm for learning nonparametric causal graphs and Optimal estimation of Gaussian DAG models leveraged this principle to design polynomial-time algorithms that reconstruct nonparametric and Gaussian DAGs. By iteratively peeling away the node with the minimum conditional variance, these algorithms reconstruct the topological order from source to sink.
More recently, Constantinos Costis Daskalakis et al. demonstrated that in equal-variance Gaussian models, this variance growth allows for the iterative identification of source nodes without requiring global matrix estimations or condition number bounds. To generalize this ordering principle, Aramayis Dallakyan et al. introduced a majorization framework, proving that the true causal ordering minimizes the dispersion of noise variances across the entire system, weakly majorizing any alternative ordering.
In real-world applications where these assumptions hold, variance-based orientation provides a direct path to structure learning. For example, Shenbao Yu et al. successfully applied variance-based causal orientation to student performance data, distinguishing between exogenous foundational skills and endogenous downstream skills based on their variance signatures.
Generalizing the Sorting Principle
The concept of sorting variables by variance-like signatures extends far beyond linear Gaussian models:
Score-Sortability
Bryon Aragam et al. discovered that the variance of a variable's score function entry (the derivative of the log-density) is proportional to its number of descendants in a causal graph. This "score-sortability" allows algorithms to recursively identify leaf nodes (variables that are effects but not causes) and reconstruct the causal order even when the model is severely misspecified or lacks Gaussian noise.
Overdispersion in Count Data
For discrete count data, Gunwoong Park et al. showed that causal directionality can be identified via overdispersion scoring. In Poisson DAG models, root nodes exhibit equal mean and variance, while downstream nodes accumulate inherited noise, causing their variance to exceed their mean.
Latent Spaces and VAEs
This variance alignment also appears in representation learning. Dominik Zietlow et al. revealed that $\beta$-VAEs do not inherently learn disentangled causal factors through structural reasoning; instead, they exploit a benchmark bias where semantic generative factors happen to align with directions of maximum statistical variance. To turn this into an explicit inductive bias, Avinash Kori et al. proposed using the variance of second-order derivatives (Hessians) of a latent distribution's log-likelihood to identify leaf nodes and enforce a causal ordering prior in unsupervised learning.
Moving Beyond Fragile Heuristics
Despite its mathematical utility, relying purely on variance sorting is fragile. As Bao Duong et al. point out, marginal variance sorting heuristics fail completely in dense graphs or when data is standardized. Similarly, Mathieu Chevalley et al. note that varsortability breaks down when the underlying noise variances are heterogeneous, prompting the need for interventional sorting methods that do not rely on passive variance gradients.
To prevent future algorithms from accidentally gaming these statistical shortcuts, the community has shifted toward artifact-neutral benchmarking. The authors of Standardizing Structural Causal Models introduced internally-standardized Structural Causal Models (iSCMs). By standardizing each variable during the generative process rather than post-hoc, iSCMs eliminate variance accumulation and $R^2$-sortability artifacts entirely.
Standard SCM Generation:
[Source Node: Var=1.0] ---> [Intermediate: Var=2.5] ---> [Sink Node: Var=5.2]
(Creates a clear variance gradient that sorting heuristics can exploit)
iSCM Generation (Artifact-Neutral):
[Source Node: Var=1.0] ---> [Standardize] ---> [Intermediate: Var=1.0] ---> [Standardize] ---> [Sink Node: Var=1.0]
(Forces algorithms to learn true structural properties rather than scale shortcuts)
By removing these incidental statistical clues, such frameworks force causal discovery models to rely on true structural principles, ensuring that high benchmark scores translate to robust performance on real-world data. To complement this, alternative evaluation paradigms, such as the interventional risk estimation framework proposed by Marco Eigenmann et al., allow researchers to validate algorithms without relying on synthetic simulations that share the same mathematical biases as the models under test.
Key Foundational Work
On causal discovery with an equal-variance assumption (2019) — contributed 3 concept elements to this direction.
Learning Large-Scale Poisson DAG Models based on OverDispersion Scoring (2015) — contributed 1 concept elements to this direction.