Program Evaluation and Causal Inference With High-Dimensional Data

Alexandre Belloni, Victor Chernozhukov, I. Fern�ndez-Val, Christian HansenEconometrica 2013

Abstract

In modern empirical research, the availability of high-dimensional data—where the number of potential control variables may exceed the sample size—presents both an opportunity and a significant statistical challenge. This paper provides a framework for efficient estimation and "honest" inference for a variety of treatment effects, including local average (LATE) and local quantile treatment effects (LQTE). By combining machine learning regularization (Lasso) with Neyman orthogonal moment conditions, the authors develop a method that remains valid even after automated variable selection. The approach is applied to the effect of 401(k) participation on accumulated assets, demonstrating its robustness in environments with over 1,700 potential control variables.

The Challenge of High-Dimensional Causal Inference

To estimate the causal effect of a treatment, researchers must account for confounders—variables that influence both the treatment assignment and the outcome. In the era of Big Data, researchers often have access to hundreds or thousands of potential controls, including complex interactions and non-linear terms.

Traditionally, researchers relied on intuition or manual selection to choose which variables to include. However, using standard machine learning tools like Lasso to automatically select the "best" controls and then running a simple regression on the survivors leads to "post-selection bias." This occurs because the selection process is inherently "greedy"; it may omit variables with small but collectively important effects on the outcome or treatment. Consequently, standard p-values and confidence intervals become invalid, failing to provide the "honest" coverage required for reliable scientific inference.

Methodology: Achieving "Honesty" through Orthogonality

The authors propose a three-step procedure designed to be uniformly valid across a wide range of models, provided the underlying structure satisfies an approximate sparsity assumption—the idea that the true relationship can be well-approximated by a small, though unknown, subset of the available variables.

1. Machine Learning for Nuisance Parameters

Instead of modeling only the outcome, the framework uses Lasso (or Post-Lasso) to estimate two distinct "nuisance" relationships:

  • The Propensity Score: How the controls predict the probability of receiving the treatment.
  • The Outcome Regression: How the controls predict the outcome variable.

2. Neyman Orthogonal Moment Conditions

The core innovation is the use of an estimating equation (a moment condition) that is "doubly robust" or Neyman orthogonal. Mathematically, the derivative of this equation with respect to the nuisance parameters is zero at the true values. This property ensures that the errors inherent in the machine learning step (Step 1) have only a second-order effect on the final treatment effect estimate. By modeling both the treatment and the outcome, the bias from the selection step effectively cancels out.

3. Functional and Quantile Inference

The framework extends beyond simple average effects to handle "function-valued" outcomes. This allows for the estimation of Quantile Treatment Effects (QTE), which reveal how a treatment impacts individuals at different points in the distribution (e.g., the 10th vs. 90th percentile). To maintain valid inference across these continuums, the authors employ a multiplier bootstrap method to generate simultaneous, honest confidence bands.

Empirical Application: 401(k) Eligibility and Wealth

The authors demonstrate the power of this method using the 1991 Survey of Income and Program Participation (SIPP) to estimate the effect of 401(k) eligibility and participation on net financial assets.

This application presents two major hurdles: wealth data is highly skewed, and participation is endogenous (individuals who choose to participate likely differ systematically from those who do not). The researchers expanded a baseline set of 35 controls into a high-dimensional set of 1,756 variables by including quadratic splines and complex interactions of income, age, and other demographic factors.

Key Findings

  • Stability: The proposed method remained stable as the control set expanded from 35 to 1,756 variables. In contrast, standard methods without selection failed in the high-dimensional setting, often producing impossible propensity scores of exactly 0 or 1.
  • Heterogeneity: The analysis of quantiles revealed that 401(k)s do not help everyone equally. The impact was moderate at lower quantiles but massive at higher quantiles, suggesting that the program has the largest effect on individuals who already possess a higher "preference for savings."
  • Simulation Performance: In synthetic tests, a "naive" post-selection estimator exhibited error rates as high as 50% for a 5% significance test. The authors' "honest" estimator consistently maintained the target 5% error rate across various scenarios.

Conclusion

This work provides a rigorous bridge between machine learning and econometrics. By leveraging Neyman orthogonality, researchers can utilize the predictive power of high-dimensional regularization while maintaining the integrity of causal inference. This framework lays the theoretical foundation for what is now widely known as Double Machine Learning, allowing for the use of even more complex algorithms (like Random Forests or Neural Networks) in the pursuit of unbiased causal discovery.

Summary updated

Abstract

The accepted manuscript version (last revised 5 Jan 2018 (v8)) has 118 pages, 3 tables, 11 figures, and includes supplementary appendix. This version corrects some typos in Example 2 of the published version. This supplement contains 11 appendices with additional results and some omitted proofs. Appendices F-J include additional results for Sections 2-7, respectively. Appendix K gathers auxiliary results on algebra of covering entropies. Appendices L and M contain the proofs of Sections 4 and 5 omitted from the main text. Appendix N contains the proofs of Sections 6 omitted from the main text, together with the proofs of the additional results for Section 6 in Appendix I. Appendix O reports the results of a simulation experiment.
BibTeX
@inproceedings{belloni2013program,
  author = {Alexandre Belloni and Victor Chernozhukov and I. Fern�ndez-Val and Christian Hansen},
  title = {{Program Evaluation and Causal Inference With High-Dimensional Data}},
  booktitle = {Econometrica},
  year = {2013},
}
Figure