Conditional Independence Testing under Model Misspecification

This direction is merged into another direction: conditional distribution estimation dependency. Its last version stays readable and citable here.

Version 1 of 112 papers18 conceptsfrozen 2026-10-04

Conditional independence (CI) testing is the bedrock of causal discovery, algorithmic fairness, and feature selection. It asks a fundamental question: Is variable $X$ independent of $Y$, once we have already accounted for the information in $Z$?

While conceptually simple, CI testing is notoriously difficult in practice. Recent research has shifted from treating it as a pure statistical problem to a machine learning challenge, using regression models to "clean" the data before testing. However, this approach introduces a dangerous new failure mode: if the regression model is even slightly wrong, the test can produce confident, false discoveries.

The Intuition: Regressing Out the Confounder

The most common way to test if $X \perp Y \mid Z$ is to "control for" $Z$ by removing its influence from both $X$ and $Y$. As Hao Zhang and Kun Zhang describe in Causal Discovery Using Regression-Based Conditional Independence Tests, we can leverage Additive Noise Models (ANMs).

The logic works like this:

  1. Regress $X$ on $Z$ to get an estimate $\hat{X} = f(Z)$.
  2. Calculate the residual $\epsilon_X = X - \hat{X}$. This residual represents the part of $X$ that cannot be explained by $Z$.
  3. Repeat for $Y$ to get $\epsilon_Y = Y - g(Z)$.
  4. Test for independence between $\epsilon_X$ and $\epsilon_Y$.

If $X$ and $Y$ are only related through $Z$, then their "unexplained" parts (the residuals) should be independent. If the residuals are still dependent, we conclude there is a direct link between $X$ and $Y$.

The Problem: Information Leakage

The "practical hardness" of this approach, as explored by Zheng He and Arthur Gretton in On the Hardness of Conditional Independence Testing In Practice, stems from the fact that our regression models are never perfect.

When a model is misspecified—meaning its inductive biases don't match the true underlying function—it fails to capture the full relationship between the variables and the confounder $Z$. This results in "information leakage." Some of $Z$'s influence remains trapped inside the residuals $\epsilon_X$ and $\epsilon_Y$.

Because both residuals now contain "leaked" bits of $Z$, they will appear statistically dependent. The independence test then incorrectly rejects the null hypothesis, leading to a false positive (Type I error). Felipe Maia Polo and Moulinath Banerjee demonstrate in Conditional independence testing under misspecified inductive biases that even minor modeling errors can cause these tests to fail catastrophically, mistaking residual noise for genuine causal relationships.

Robust Solutions: Kernels and Permutations

To combat this sensitivity, researchers have developed more robust frameworks that move beyond simple linear residuals.

Feature-to-Feature Regression

Qinyi Zhang and Dino Sejdinovic introduced KRESIT in Feature-to-Feature Regression for a Two-Step Conditional Independence Test. Instead of regressing raw values, they perform regression in a Reproducing Kernel Hilbert Space (RKHS). By mapping the data into a high-dimensional feature space, the model can capture complex, non-additive dependencies that standard regression might miss, thereby reducing the leakage that leads to false discoveries.

Conditional Permutation

Another strategy is to change how we determine "significance." Standard tests often compare the test statistic to a theoretical distribution that assumes a perfect model. In The conditional permutation test, Thomas B. Berrett and Richard J. Samworth propose the Conditional Permutation Test (CPT). Rather than generating new synthetic samples (which requires a perfect model of $P(X|Z)$), the CPT non-uniformly shuffles existing data points. This provides a more robust safeguard against model inaccuracies.

Localized and Power-Based Testing

For more complex data types, such as the functional data (like EEG signals) discussed by Anton Rask Lundborg and Rajen D. Shah in Conditional Independence Testing in Hilbert Spaces, the requirements for regression accuracy become even stricter.

To address this, Wenjie Wang and Bo Han developed PowerKCI, a method that optimizes kernel selection specifically to maximize the "test power" while accounting for regression bias. Similarly, Xiaoyang Wu and Changliang Zou proposed using localized conformal p-values in Conditional Testing based on Localized Conformal p-values to ensure that error control holds even when the underlying machine learning models are complex or partially misspecified.

Summary

The shift in the field is clear: we can no longer assume our predictive models are "correct." Modern CI testing focuses on decoupling the test's validity from the model's accuracy. By using kernel-based feature mapping, clever permutation strategies, and localized error control, researchers are building tests that can distinguish between a genuine causal link and the mere "echo" of a poorly modeled confounder.

Papers in this version

History & citationVersion 1 · 2026-10-04

Merged into another direction · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_conditional_independence_testing_under_m_32c99956,
  title = {Conditional Independence Testing under Model Misspecification},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/conditional-independence-testing-under-model-misspecification/rdr_32c99956a2c4459c9d940f68caeb255a},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_conditional_independence_testing_under_m_32c99956_v1,
  title = {Conditional Independence Testing under Model Misspecification},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/conditional-independence-testing-under-model-misspecification/rdr_32c99956a2c4459c9d940f68caeb255a/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}