Conditional Independence Testing via Residual Analysis

This direction is merged into another direction: residual-based conditional independence testing. Its last version stays readable and citable here.

Version 1 of 121 papers31 conceptsfrozen 2026-10-04

Determining whether two variables $X$ and $Y$ are independent given a third variable $Z$ is a fundamental challenge in causal discovery and machine learning. In theory, this requires checking if $P(X, Y | Z) = P(X | Z)P(Y | Z)$ for every possible value of $Z$. In practice, especially when $Z$ is high-dimensional or continuous, we rarely see the same value of $Z$ twice, making direct conditioning impossible.

A growing body of research addresses this "curse of dimensionality" by transforming the problem: instead of testing independence within slices of $Z$, we "subtract" the influence of $Z$ from both $X$ and $Y$ and test the independence of what remains. These "leftovers" are known as residuals.

The Core Intuition: Subtracting the Confounder

The central idea is to decouple the modeling of the conditioning variable $Z$ from the testing of the relationship between $X$ and $Y$. If $Z$ is the only reason $X$ and $Y$ appear related, then once we account for $Z$'s influence, the remaining variation in $X$ should have nothing to do with the remaining variation in $Y$.

As Hao Zhang, Kun Zhang, and colleagues demonstrate in Causal Discovery Using Regression-Based Conditional Independence Tests, we can use nonlinear regression to model $X = f(Z) + \epsilon_X$ and $Y = g(Z) + \epsilon_Y$. If $X$ and $Y$ are conditionally independent given $Z$, then the residuals $\epsilon_X$ and $\epsilon_Y$ should be independent of each other. This shifts the difficulty from a complex conditional test to a simpler marginal independence test between two variables.

Beyond Linear Residuals

While simple subtraction works for additive noise, real-world data is often more complex. Researchers have developed more sophisticated ways to "distill" the information in $Z$:

Computational Efficiency and Distillation

Traditional conditional tests are often slow because they require repeated model fitting or complex permutations. The residual-based approach offers a significant speedup by "distilling" the nuisance variables once.

Molei Liu, Aaditya Ramdas, and colleagues formalize this in Fast and Powerful Conditional Randomization Testing via Distillation. By distilling the effect of $Z$ into low-dimensional residuals, they can perform randomization tests much faster than traditional methods. Similarly, the FRCIT method described in Testing Independence Between Linear Combinations for Causal Discovery achieves massive speedups by focusing on the statistical moments (like kurtosis) of linear residuals rather than expensive kernel computations.

Why It Works: The Power of Decoupling

The success of this direction stems from a simple trade-off: it replaces a hard statistical problem (conditional testing) with a well-understood engineering problem (regression or density estimation).

As noted by Anton Rask Lundborg and Rajen D. Shah in their work on the Generalised Hilbertian Covariance Measure (GHCM), while CI testing is "fundamentally impossible" without assumptions, residual-based methods provide a practical path forward. By leveraging modern machine learning to "clean" the data of confounding influences, we can isolate the specific signals needed to build accurate causal graphs and fair representations. This is exemplified by CIRCE, a method by Roman Pogodin and Arthur Gretton in Efficient Conditionally Invariant Representation Learning, which uses residual covariance to help neural networks ignore distractor variables.

Papers in this version

History & citationVersion 1 · 2026-10-04

Merged into another direction · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_conditional_independence_testing_via_res_40cfa185,
  title = {Conditional Independence Testing via Residual Analysis},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/conditional-independence-testing-via-residual-analysis/rdr_40cfa185acb54fabaef8a42bb5560608},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_conditional_independence_testing_via_res_40cfa185_v1,
  title = {Conditional Independence Testing via Residual Analysis},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/conditional-independence-testing-via-residual-analysis/rdr_40cfa185acb54fabaef8a42bb5560608/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}