On the power of conditional independence testing under model-X

Eugene Katsevich, Aaditya Ramdas

Abstract

For testing conditional independence (CI) of a response $Y$ and a predictor $X$ given covariates $Z$, the recently introduced model-X (MX) framework has been the subject of active methodological research, especially in the context of MX knockoffs and their successful application to genome-wide association studies. In this paper, we build a theoretical foundation for the MX CI problem, yielding quantitative explanations for empirically observed phenomena and novel insights to guide the design of MX methodology. We focus our analysis on the conditional randomization test (CRT), whose validity conditional on $Y,Z$ allows us to view it as a test of a point null hypothesis involving the conditional distribution of $X$. We use the Neyman-Pearson lemma to derive the most powerful CRT statistic against a point alternative as well as an analogous result for MX knockoffs. We define CRT-style analogs of $t$- and $F$-tests with explicit critical values, and show that they have uniform asymptotic Type-I error control under the assumption that only the first two moments of $X$ given $Z$ are known, a significant relaxation of MX. We derive expressions for the power of these tests against local semiparametric alternatives using Le Cam's local asymptotic normality theory, explicitly capturing the prediction error of the underlying learning algorithm. Finally, we pave the way for estimation in the MX setting by drawing connections to semiparametric statistics and causal inference. Thus, this work forms explicit bridges from MX to both classical statistics (testing) and modern causal inference (estimation).

Introduction

Conditional Independence (CI) testing is a cornerstone of modern statistical inference, particularly in high-dimensional settings like genome-wide association studies (GWAS) or neuroscience. The goal is to determine if a specific variable $X$ (e.g., a genetic variant) influences a response $Y$ (e.g., a disease phenotype) after accounting for a set of confounding covariates $Z$. Formally, we test the null hypothesis $H_0: X \perp Y | Z$.

Historically, researchers relied on "Model-Y" assumptions, which require specifying the functional relationship between the response and the predictors. However, in the era of big data, we often possess extensive unlabeled data regarding the predictors ($X$ and $Z$) but limited labeled data for the response ($Y$). This shift led to the "Model-X" (MX) framework. Unlike traditional methods, MX assumes the conditional distribution of the predictors ($X|Z$) is known or can be accurately estimated, while making no assumptions about the relationship between $X$ and $Y$. While MX methods like the Conditional Randomization Test (CRT) and Knockoffs have gained empirical popularity, their theoretical underpinnings remained largely unexplored until now.

Theoretical Framework: The CRT as a Point Null

The authors provide a rigorous foundation for MX by focusing on the Conditional Randomization Test (CRT). The key theoretical breakthrough involves conditioning on the observed values of $Y$ and $Z$. By doing so, the complex composite null hypothesis of conditional independence is transformed into a simpler point null hypothesis regarding the distribution of $X$.

This shift allows the application of classical statistical tools, most notably the Neyman-Pearson Lemma. The authors prove that the most powerful CRT statistic against a specific alternative is the likelihood ratio of the response given the predictors. This result justifies the common practice of using sophisticated machine learning models (e.g., Random Forests or Deep Learning) to estimate these ratios, providing a theoretical "green light" for integrating black-box algorithms into rigorous statistical tests.

Robustness and the MX(2) Assumption

A common critique of the Model-X framework is the requirement of knowing the full conditional distribution of $X|Z$, which is often unrealistic in practice. This paper introduces the MX(2) assumption—a significant relaxation that requires knowing only the first two conditional moments (mean and variance) of $X|Z$.

The authors demonstrate that CRT-style analogs of classical $t$- and $F$-tests maintain uniform asymptotic Type-I error control under this relaxed assumption. This finding suggests that MX methods are far more robust than previously thought, as they remain valid in large samples even if the higher-order moments of the predictor distribution are misspecified.

Quantifying Power and Prediction Error

One of the most significant contributions of this work is the explicit quantification of how machine learning performance impacts statistical power. Using Le Cam’s Local Asymptotic Normality (LAN) theory, the authors derive an asymptotic power formula for a 1D predictor:

$$\lim_{n \to \infty} \mathbb{E}[\pi_n] = \Phi \left( z_\alpha + \frac{h \sqrt{\mathbb{E}[\text{Var}[X|Z]]}}{\sqrt{\sigma^2 + \mathcal{E}(\hat{g})}} \right)$$

In this expression, $\mathcal{E}(\hat{g})$ represents the Mean Squared Error (MSE) of the machine learning model used to predict $Y$. This formula provides a mathematical proof for the "folk wisdom" that better prediction leads to better testing: the prediction error acts as additional noise, directly reducing the effective signal-to-noise ratio. Furthermore, the authors prove that the CRT is consistent—it will eventually detect any real effect as sample size increases—regardless of the ML model's quality, provided the model is trained on independent data.

Bridging to Causal Inference

Beyond testing, the paper establishes a bridge to modern causal inference. By treating the Model-X assumption as a known "propensity function," the authors show that observational studies can be treated similarly to randomized experiments. This allows for the estimation of effects (such as dose-response functions) using semiparametric statistics, moving the Model-X framework from a purely testing-oriented tool to a comprehensive framework for estimation and inference.

Conclusion and Future Directions

This research provides the first comprehensive theoretical treatment of the Model-X framework, confirming its optimality, robustness, and efficiency. While the current theory assumes a fixed number of covariates $p$ as $n \to \infty$, it sets the stage for future work in high-dimensional asymptotics ($p > n$) and the analysis of "in-sample" training where data splitting is not utilized. By grounding MX in classical theory, this work ensures that these modern methods are as statistically sound as they are computationally powerful.

Summary updated

Abstract

For testing conditional independence (CI) of a response Y and a predictor X given covariates Z, the recently introduced model-X (MX) framework has been the subject of active methodological research, especially in the context of MX knockoffs and their successful application to genome-wide association studies. In this paper, we study the power of MX CI tests, yielding quantitative insights into the role of machine learning and providing evidence in favor of using likelihood-based statistics in practice. Focusing on the conditional randomization test (CRT), we find that its conditional mode of inference allows us to reformulate it as testing a point null hypothesis involving the conditional distribution of X. The Neyman-Pearson lemma then implies that a likelihood-based statistic yields the most powerful CRT against a point alternative. We also obtain a related optimality result for MX knockoffs. Switching to an asymptotic framework with arbitrarily growing covariate dimension, we derive an expression for the limiting power of the CRT against local semiparametric alternatives in terms of the prediction error of the machine learning algorithm on which its test statistic is based. Finally, we exhibit a resampling-free test with uniform asymptotic Type-I error control under the assumption that only the first two moments of X given Z are known, a significant relaxation of the MX assumption.
BibTeX
@inproceedings{katsevich2022power,
  author = {Eugene Katsevich and Aaditya Ramdas},
  title = {{On the power of conditional independence testing under model-X}},
  booktitle = {Electronic},
  year = {2022},
  doi = {10.1214/22-ejs2085},
}
Figure