Abstract
We investigate the relationship between conditional independence (CI) $x \perp y | Z$ and the independence of two residuals $x – E(x | Z) \perp y – E(y | Z)$, where $x$ and $y$ are two random variables, and $Z$ is a set of random variables. We show that if $x, y$ and $Z$ are generated by following linear structural equation models and all external influences follow Gaussian distributions, then $x \perp y | Z$ if and only if $x – E(x | Z) \perp y – E(y | Z)$. Furthermore, if all these external influences follow non-Gaussian distributions and the model satisfies the structural faithfulness condition, then we have $x \perp y | Z \Leftrightarrow x – E(x | Z) \perp y – E(y | Z)$. We apply these results to the causal discovery problem, showing that in linear non-Gaussian contexts, residuals can help distinguish Markov equivalence classes. Compared with existing discretization-based and kernel-based CI testing methods, the proposed ReCIT method provides a simpler, more robust way to measure CI, requiring only one unconditional independence test and two regression operations.
Introduction
In causal discovery, determining whether two variables are independent given a third set ($x \perp y | Z$) is a fundamental yet notoriously difficult task. If $X$ and $Y$ become independent once $Z$ is accounted for, it suggests the absence of a direct causal link. However, standard conditional independence (CI) testing suffers from the "curse of dimensionality": as the size of the conditioning set $Z$ grows, the data requirements for accurate testing increase exponentially. Existing kernel-based methods, while powerful, are computationally expensive and sensitive to noise. This paper introduces a simplified approach that reduces complex conditional tests to basic marginal independence tests between residuals.
The ReCIT Method
The proposed method, ReCIT (Residual-based Conditional Independence Test), simplifies CI testing through three primary steps:
- Regression of $x$ on $Z$: Obtain the residual $\epsilon_x = x - E(x|Z)$.
- Regression of $y$ on $Z$: Obtain the residual $\epsilon_y = y - E(y|Z)$.
- Independence Testing: Test if $\epsilon_x$ and $\epsilon_y$ are marginally independent.
Theoretically, the authors prove that under linear structural equation models (SEMs), $x \perp y | Z$ is equivalent to the independence of these residuals. This equivalence holds for both Gaussian and non-Gaussian noise. By integrating ReCIT into the standard PC algorithm (PCReCIT), the authors can orient edges in chains and forks by exploiting the asymmetry of non-Gaussian residuals—a capability traditional constraint-based methods lack.
Experimental Results
The authors evaluated ReCIT against the Kernel-based Conditional Independence Test (KCIT) using simulated data and benchmark causal networks like Hailfinder and Win95pts.
Accuracy and Dimensionality
A key advantage of ReCIT is its stability as the dimensionality of the conditioning set $Z$ increases. While KCIT's Type I error rate climbs significantly with more variables in $Z$, ReCIT maintains a stable error rate near the significance level (0.05).
Causal Orientation in Non-Gaussian Settings
In non-Gaussian environments, residuals carry "fingerprints" of causal direction. If $Z$ causes $X$, the residual of $X$ given $Z$ is independent of $Z$. This property allows PCReCIT to orient edges that are typically indistinguishable in Markov equivalence classes. On a 6-node graph with no V-structures, PCReCIT correctly oriented all causal directions, whereas the kernel-based approach left most edges undirected.
Performance Summary
Experiments also demonstrated that PCReCIT outperforms PCKCIT in discovering the "skeleton" of causal graphs, particularly with small sample sizes ($n < 100$). Furthermore, ReCIT showed greater robustness to noise, remaining accurate even when the noise-to-signal ratio was low.
Conclusion
ReCIT provides a simpler and more stable alternative to existing CI testing methods by reducing conditional tests to marginal ones. This simplification avoids complex conditional density estimations, making it particularly effective for high-dimensional data. While currently limited to linear models and reliant on the faithfulness assumption, future work could extend this residual-based framework to non-linear relationships using advanced regression techniques like Gaussian Process regression.
Summary updated