When estimating causal effects from observational data, we rarely know the exact data-generating process. To adjust for confounding variables, researchers typically rely on auxiliary models known as nuisance parameters. These include propensity scores (which model the probability of treatment assignment) and outcome regressions (which model the expected outcome given covariates).
The fundamental challenge is that these nuisance models are often misspecified. If we assume a linear relationship when the true mechanism is highly non-linear, our final causal estimates become biased and unreliable. This article explains how modern causal inference addresses this fragility through the principles of double robustness and Neyman orthogonality.
The Fragility of Plug-in Estimators
Traditional "plug-in" estimators insert nuisance parameter estimates directly into a causal formula. For example, an inverse probability weighting (IPW) estimator relies entirely on the correctness of the propensity score model. If the functional form of this model is even slightly incorrect, the resulting causal estimate can be severely biased.
As noted by Eric J. Tchetgen Tchetgen in Semiparametric theory for causal mediation analysis: Efficiency bounds, multiple robustness and sensitivity analysis, relying on rigid parametric forms to account for numerous confounders leaves causal estimates highly vulnerable to functional form misspecification. When the true underlying data-generating process deviates from these assumed structures, standard adjustments can perform worse than unadjusted comparisons, a phenomenon analyzed by Nathan Kallus in Causal Inference by Minimizing the Dual Norm of Bias: Kernel Matching & Weighting Estimators for Causal Effects.
The Insurance Policy of Double Robustness
To protect against misspecification, researchers design doubly robust estimators. Instead of betting everything on a single nuisance model, a doubly robust estimator combines both an outcome regression model ($\hat{\mu}$) and a propensity score model ($\hat{e}$).
The mathematical intuition behind why this works lies in the structure of the estimator's bias. The bias of a doubly robust estimator is proportional to the product of the errors of its constituent models:
$$\text{Bias} \propto (\hat{\mu} - \mu) \times (\hat{e} - e)$$
Where $\mu$ and $e$ are the true, unknown functions. If the propensity score model is perfectly specified ($\hat{e} = e$), the second term becomes zero, eliminating the bias even if the outcome model is completely wrong. Conversely, if the outcome model is correct ($\hat{\mu} = \mu$), the first term becomes zero, neutralizing any errors in the propensity score.
As discussed by Elias Bareinboim et al. in Unified Covariate Adjustment for Causal Inference, this property is vital because achieving perfect specification for any single auxiliary model is rare in high-dimensional, real-world applications.
Neyman Orthogonality and Double Machine Learning
If parametric models are prone to misspecification, a natural alternative is to use flexible machine learning (ML) models, such as deep neural networks or random forests. However, flexible models introduce a new problem: they converge to the truth much slower than standard parametric models, typically at a rate slower than $\sqrt{n}$ (e.g., $n^{-1/4}$). This slow convergence introduces a first-order regularization bias into the causal estimate, destroying standard statistical inference.
To solve this, Double Machine Learning (DML) constructs Neyman-orthogonal score functions. Imagine walking along a mountain ridge where the peak represents the true causal effect, and the horizontal axes represent errors in our nuisance models. In a standard plug-in estimator, the ridge is sloped; a small error in a nuisance model shifts your position down the slope, changing your causal estimate significantly.
An orthogonal estimator flattens the peak. It ensures that the derivative of the target causal parameter with respect to the nuisance parameters is exactly zero at the true value. Because the slope is flat, small errors or slow convergence in the machine learning models only cause a negligible, second-order shift in the causal estimate.
Jin Tian demonstrates this property in Estimating Identifiable Causal Effects through Double Machine Learning, showing how orthogonal scores allow researchers to use arbitrary ML models while maintaining root-$n$ convergence and valid confidence intervals. However, the authors of Orthogonal Machine Learning: Power and Limitations. caution that this robustness has limits: if the underlying ML models converge exceptionally slowly (worse than $n^{-1/4}$), the first-order bias can reappear.
Scaling to Complex Causal Environments
The principles of double robustness and Neyman orthogonality are not restricted to simple average treatment effects; they have been extended to handle complex structural challenges.
Unknown Graphs and Hidden Confounding
When the exact causal relationships are unknown, researchers must infer them from data. Jin Tian et al. in Estimating Identifiable Causal Effects on Markov Equivalence Class through Double Machine Learning extend orthogonal learning to cases where only a Markov Equivalence Class can be identified, protecting the effect estimation from the dual errors of graph mis-identification and nuisance parameter bias. For settings with unobserved confounders, AmirEmad Ghassami develops a non-parametric minimax kernel framework in Minimax Kernel Machine Learning for a Class of Doubly Robust Functionals with Application to Proximal Causal Inference to robustly solve the integral equations required for proximal causal inference.
Beyond Averages to Full Distributions
Often, decision-makers need to understand the full range of risks rather than just the average outcome. To address this, Valentyn Melnychuk et al. introduce Interventional Normalizing Flows in Normalizing Flows for Interventional Density Estimation, combining deep generative architectures with semi-parametric efficiency bounds to predict entire potential outcome distributions robustly.
Policy Adaptation and Covariate Shift
When deploying causal models to new populations, the distribution of covariates often shifts. In Optimal Policy Adaptation Under Covariate Shift, Peng Wu leverages doubly robust structures to identify optimal treatment policies even when the target population differs significantly from the training data, ensuring that misspecified density ratios do not lead to harmful policy decisions.