Abstract
This paper introduces Kernel Anchor Regression (KAR), a novel framework that extends anchor regression to capture nonlinear causal relationships. By leveraging kernel methods and a specialized three-stage estimation procedure, KAR can identify causal effects even when hidden confounders are present. Our approach outperforms traditional linear models and existing nonlinear instrumental variable methods in both accuracy and robustness to distribution shifts.
Introduction
Determining causal relationships is a fundamental challenge in science, particularly when confounders—unobserved variables influencing both the cause ($X$) and the effect ($Y$)—are present. Standard Ordinary Least Squares (OLS) fails in these scenarios, and while Instrumental Variable (IV) methods can help, they require a "perfect" instrument. Anchor Regression (AR) serves as a middle ground, using an "anchor" variable ($Z$) to provide solutions robust against partially observed confounders. However, original AR assumes linearity, which is rarely the case in complex systems like biology or economics. This paper bridges this gap by developing a nonparametric, kernel-based version of anchor regression.
The KAR Framework
The core innovation of Kernel Anchor Regression (KAR) is the mapping of variables into a high-dimensional feature space, specifically a Reproducing Kernel Hilbert Space (RKHS). In this space, complex nonlinear relationships can be treated as linear operators.
The authors propose a three-stage estimation procedure to ensure theoretical convergence:
- Stage 1: Estimate the prediction of the cause $X$ from anchor $Z$ in the feature space.
- Stage 2: Estimate the prediction of the effect $Y$ from anchor $Z$ in the feature space.
- Stage 3: Perform a "projection-adjusted" regression to identify the causal mapping from $X$ to $Y$.
By tuning a regularization parameter $\gamma$, KAR can adapt its behavior, ranging from a nonlinear OLS equivalent ($\gamma=1$) to a nonlinear IV estimator ($\gamma \to \infty$).
Experimental Evaluation
The authors evaluated KAR using synthetic nonlinear Structural Equation Models (SEM) and the real-world Smoking Dataset (NMES).
Key Results
- Superior Accuracy: In synthetic tests, KAR achieved significantly lower Mean Squared Error (MSE) compared to linear AR, standard Kernel IV (KIV), and Kernel Ridge Regression.
- Out-of-Distribution (OOD) Robustness: KAR is designed for invariance. When the distribution of the anchor variable $Z$ was shifted between training and testing, KAR's prediction error remained the most stable among all tested methods.
- Real-World Application: On the NMES dataset, KAR successfully captured a nonlinear trend where medical expenditure increases significantly at lower smoking amounts and plateaus at higher levels—a nuance missed by linear models.
Conclusion and Future Work
KAR provides a powerful, flexible tool for causal discovery in nonlinear systems. While it demonstrates significant improvements in accuracy and robustness, future work is needed to develop data-adaptive methods for selecting the anchor parameter $\gamma$ and to address the computational scaling inherent in kernel methods.
Summary updated