Abstract
Chernozhukov, Chetverikov, Demirer, Duflo, Hansen, and Newey (2016) provide a generic double/de-biased machine learning (DML) approach for obtaining valid inferential statements about focal parameters, using Neyman-orthogonal scores and cross-fitting, in settings where nuisance parameters are estimated using a new generation of nonparametric fitting methods for high-dimensional data, called machine learning methods. In this note, we illustrate the application of this method in the context of estimating average treatment effects (ATE) and average treatment effects on the treated (ATTE) using observational data.
Introduction
In modern causal analysis, researchers often seek to estimate a specific "focal parameter"—such as the impact of a policy or a medical intervention—while controlling for a high-dimensional set of confounding variables. While modern machine learning (ML) algorithms like Random Forests, Gradient Boosting, and Neural Networks excel at predictive tasks, they are notoriously difficult to use for statistical inference (calculating p-values and confidence intervals).
Directly plugging ML predictions into standard statistical formulas leads to biased results. The Double Machine Learning (DML) framework addresses this by combining Neyman-orthogonal scores with cross-fitting to remove the biases introduced by ML regularization and overfitting.
The Challenges of ML in Inference
Two primary sources of bias prevent the naive use of ML for causal inference:
- Regularization and Model Selection Bias: To achieve high predictive accuracy, ML models often employ regularization (like Lasso) or pruning (like Trees). This process can "shrink" coefficients or ignore variables that are crucial for unbiased treatment effect estimation, even if they are important for prediction.
- Overfitting Bias: If the same data is used to learn the structure of the model (the nuisance parameters) and to estimate the treatment effect, the noise captured by the model "leaks" into the final estimate. This makes the treatment effect appear more precise than it actually is, leading to invalid confidence intervals.
The Double Machine Learning (DML) Solution
The DML approach provides a robust "recipe" for inference using two fundamental pillars:
1. Neyman-Orthogonal Scores
Instead of a simple regression, DML utilizes specific moment conditions, or "scores," that are mathematically designed to be insensitive to small errors in the nuisance parts of the model (such as the propensity score or the baseline outcome regression). By ensuring the score is "orthogonal" to the nuisance parameters, the final treatment effect estimate remains robust even when the underlying ML models contain the inevitable errors associated with high-dimensional estimation.
2. K-fold Cross-Fitting
To eliminate overfitting bias, DML employs a sample-splitting technique known as cross-fitting. The data is split into $K$ parts. The nuisance parameters are trained on $K-1$ parts, and the treatment effect is estimated on the remaining part. This process is rotated across all folds and averaged. This ensures that the "noise" learned by the ML model during the training phase does not contaminate the treatment effect estimation phase.
Empirical Applications
The authors demonstrate the versatility of DML through two distinct empirical studies:
401(k) Eligibility and Wealth
This observational study examined the effect of 401(k) eligibility on net financial assets. This is a high-dimensional problem because many variables (income, age, education, family size) must be controlled for to isolate the treatment effect.
- Results: Without controls, the raw difference in assets was $19,559. After applying DML to control for confounders, the estimated ATE dropped to a range of **$7,000 to $9,000**. The results remained remarkably stable across different ML algorithms (Lasso vs. Random Forest), providing high confidence in the findings.
Pennsylvania Reemployment Bonus Experiment
This study utilized data from a randomized control trial (RCT) to investigate how cash bonuses affect the duration of unemployment. Even in an RCT, DML can be used to control for covariates to increase the precision of the estimate.
- Results: The ATE of the cash bonus on the log duration of unemployment was consistently estimated at approximately -0.07 to -0.08. This indicates the bonus successfully reduced the time spent unemployed, and the DML framework provided more precise estimates (lower standard errors) than simpler linear models.
Key Insights and Best Practices
- The Necessity of Both Pillars: The authors emphasize that "Double" is the critical word; one must use both orthogonality (to fix regularization bias) and cross-fitting (to fix overfitting). Relying on only one is insufficient for valid inference.
- Method Agnosticism: DML works with almost any high-quality ML method. Whether using Lasso, Gradient Boosting, or Ensembles (stacking multiple models), the framework produces broadly consistent results.
- Sample Splitting Sensitivity: While DML is asymptotically valid, results in smaller datasets can be sensitive to how the data is split. The authors recommend "repeated cross-fitting"—taking the median estimate across multiple random splits—to ensure maximum robustness.
Conclusion
Double Machine Learning bridges the gap between the predictive power of modern ML and the rigorous requirements of causal inference. By providing a generic framework that is agnostic to the specific ML algorithm used, it allows researchers to leverage the most powerful predictive tools available while still obtaining reliable, de-biased estimates of treatment effects.
Summary updated