The gold standard for estimating the causal effect of a treatment on an outcome is to control for all variables that simultaneously influence both. In observational studies, this is known as the backdoor criterion or the assumption of no unmeasured confounding. But in the real world, the most critical confounders—such as a patient's baseline health status, a consumer's underlying motivation, or socio-economic nuances—are often unobserved. When these latent factors are ignored, standard statistical models yield biased, misleading results.
For a long time, researchers relied on instrumental variables (IVs) to bypass hidden confounding. However, valid instruments must satisfy strict "exclusion restrictions"—meaning they must affect the outcome only through the treatment—which are notoriously difficult to verify or find in practice.
As Alexander Nicholas D'Amour demonstrated in On Multi-Cause Approaches to Causal Inference with Unobserved Counfounding: Two Cautionary Failure Cases and A Promising Alternative., simply observing multiple causes or complex data patterns is not a silver bullet; without explicit structural assumptions or auxiliary data, causal effects remain fundamentally unidentifiable.
Proximal causal inference provides a powerful alternative. Instead of requiring a perfect instrument or complete measurement of every confounder, it leverages proxy variables—imperfect, noisy "sensors" of the hidden world—to mathematically reconstruct and subtract confounding bias.
The Intuition: Why One Proxy Fails But Two Succeed
To understand why proximal causal inference works, we must first understand why a single proxy variable is insufficient. Suppose we want to measure the effect of a treatment $A$ on an outcome $Y$, but both are influenced by an unobserved confounder $U$. We might observe a proxy variable $W$ (such as a noisy test score or a demographic indicator) that is correlated with $U$.
If we simply plug $W$ into our regression model as a substitute for $U$, our estimate will suffer from attenuation bias. Because $W$ is a noisy measurement, it cannot fully "absorb" the confounding effect of $U$. We cannot distinguish whether a change in $W$ reflects a true shift in the hidden confounder $U$ or just random measurement error.
The breakthrough of proximal causal inference is the dual-proxy mechanism. By introducing a second, independent proxy variable, we can untangle the noise from the signal. The framework divides proxies into two distinct teams:
- Outcome Proxies ($W$): Variables that are strongly related to the hidden confounder $U$ but do not directly cause the treatment $A$.
- Treatment Proxies ($Z$): Variables that are strongly related to the hidden confounder $U$ but do not directly cause the outcome $Y$ (except via $A$ or $U$).
[Z: Treatment Proxy]
^
|
[U: Hidden Confounder] ---> [A: Treatment] ---> [Y: Outcome]
| ^
v |
[W: Outcome Proxy] ---------------------------------+
In Semiparametric Proximal Causal Inference, Hongming Pu et al. formalize how this dual structure allows for valid causal identification. The core idea is to find an outcome bridge function. This function maps the observed outcome proxy $W$ and the treatment $A$ to the true counterfactual outcome space.
Mathematically, we look for a function $h(W, A)$ that satisfies a conditional moment restriction: when we look across different levels of the treatment proxy $Z$, the variation in $h(W, A)$ perfectly balances out the confounding influence of $U$. Because $Z$ and $W$ are conditionally independent given $U$, any shared variation between them must be mediated by the hidden confounder. By forcing the bridge function to satisfy this relationship, we effectively "solve" for the latent confounder's impact without ever measuring it directly.
Scaling to Non-Linear Relationships with Machine Learning
Early iterations of proximal causal inference relied heavily on linear models or simple categorical data. However, real-world proxies—such as text data, satellite imagery, or high-dimensional clinical metrics—exhibit highly complex, non-linear relationships with latent variables.
To bridge this gap, researchers have fused proximal causal inference with modern machine learning tools:
- Kernel Methods: In Proximal Causal Learning with Kernels: Two-Stage Estimation and Moment Restriction, Afsaneh Mastouri et al. introduced the Kernel Proxy Variable (KPV) and Proxy Maximum Moment Restriction (PMMR) frameworks. By leveraging the flexibility of Reproducing Kernel Hilbert Spaces (RKHS), these methods solve the underlying bridge equations across infinite-dimensional feature spaces, capturing intricate, non-linear confounding patterns.
- Heterogeneous Treatment Effects: Estimating how a treatment affects different individuals (Conditional Average Treatment Effects, or CATE) is crucial for personalized medicine and policy design. Erik Sverdrup et al. developed the P-learner in Proximal Causal Learning of Conditional Average Treatment Effects. They proved that if the underlying bridge functions are accurately estimated using standard loss functions, proxy-based frameworks can achieve performance levels comparable to an "oracle" model that has direct access to the hidden confounders.
- Optimal Policy Regimes: Beyond merely estimating effects, Tao Shen et al. extended this framework to decision-making in Optimal Treatment Regimes for Proximal Causal Learning. By combining both treatment-inducing and outcome-inducing confounding bridges, the authors created a robust mechanism for learning optimal, individualized treatment rules directly from confounded observational data.
Proximal Learning in Sequential Environments
The utility of proximal causal inference extends naturally to reinforcement learning (RL) and Partially Observable Markov Decision Processes (POMDPs). In these settings, an agent interacts with an environment where the true state of the world is hidden, acting as a latent confounder that influences both the agent's historical actions and the subsequent rewards.
When evaluating a new policy using historical data (Off-Policy Evaluation, or OPE), unobserved states can create dangerous illusions of success. Proximal causal inference resolves this by treating historical and current observations as proxies for the underlying latent state.
In Pessimism in the Face of Confounders: Provably Efficient Offline Reinforcement Learning in Partially Observable Markov Decision Processes, Miao Lu introduced the Proxy variable Pessimistic Policy Optimization (P3O) algorithm. P3O uses observed trajectories as proxies to construct a conservative lower bound on policy performance, preventing the agent from overestimating the value of actions that were merely correlated with favorable hidden states in the training data.
Similarly, Rui Miao et al. tackled continuous state spaces in Off-Policy Evaluation for Episodic Partially Observable Markov Decision Processes under Non-Parametric Models. The authors integrated proxy variables as negative controls directly within recursive Bellman equations. This allowed them to frame OPE as a sequential non-parametric instrumental variable problem, providing the first finite-sample error bounds for continuous, confounded POMDPs.
By shifting the requirement from perfect measurement to complementary observation, proximal causal inference turns messy, proxy-rich datasets into reliable engines for causal discovery and decision-making.