Supplementary Appendix for "Inference on Treatment Effects After Selection Amongst High-Dimensional Controls"

Alexandre Belloni, Victor Chernozhukov, Christian HansenarXiv 2013 · DOI · arXiv

Abstract

In modern empirical research, the number of potential control variables often exceeds the sample size. While Lasso-based methods are highly effective for prediction, their naive application in estimating treatment effects can lead to severe bias and invalid inference. This paper summarizes the technical advancements and extensive simulation evidence for the "Post-Double-Selection" method, which ensures robust, "honest" inference by selecting variables from both the outcome and treatment equations.

Introduction

The advent of high-dimensional data has transformed the landscape of causal inference. Researchers frequently face the challenge of selecting a parsimonious set of controls from a vast pool of potential covariates ($p \gg n$). A common, yet flawed, approach is "Post-Single-Selection": using a selection method like Lasso to identify variables that predict the outcome and then performing ordinary least squares (OLS) on the treatment and the selected controls.

As Belloni, Chernozhukov, and Hansen demonstrate in this supplementary appendix, this single-selection approach is prone to a hidden danger: omitted variable bias. If a variable is weakly correlated with the outcome but strongly correlated with the treatment, Lasso will likely exclude it, leading to a breakdown of standard inferential procedures and erratic estimator distributions.

The Failure of Single Selection

The fundamental issue with single-selection estimators is their "non-regularity." In settings where some coefficients are small but non-zero (often referred to as the "close-to-zero" parameter sequence), the distribution of the treatment effect estimator becomes unstable. This leads to confidence intervals that fail to achieve their nominal coverage. In extreme cases, simulations show that a 95% confidence interval derived from a single-selection model may only contain the true parameter 0% to 20% of the time. This "super-efficiency" trap achieves lower variance only by sacrificing the validity of the resulting inference.

The Post-Double-Selection Method

To solve this, the authors propose a "Double Selection" framework. The procedure is designed to "immunize" the treatment effect estimate against the exclusion of important confounders by framing the problem through the lens of moment conditions. The process follows four key steps:

  1. Outcome Selection: Apply a selection method (e.g., Lasso or a conservative t-test) to identify covariates $x$ that predict the outcome $y$.
  2. Treatment Selection: Apply a selection method to identify covariates $x$ that predict the treatment $d$.
  3. The Union: Take the union of the variables selected in both steps.
  4. Final Estimation: Estimate the treatment effect by regressing $y$ on $d$ and the combined set of selected covariates.

By including variables that predict the treatment, the researcher ensures that any covariate capable of inducing significant bias—even if its direct effect on the outcome is small—is controlled for in the final model. This makes the estimator first-order insensitive to small errors in variable selection.

Technical Extensions and Robustness

The appendix extends this logic beyond simple linear models to address more complex research designs:

  • Heterogeneous Treatment Effects: The authors utilize Hahn’s (1998) efficient moment conditions to estimate Average Treatment Effects (ATE) and Average Treatment Effects on the Treated (ATT) when the treatment effect varies across individuals.
  • Split-Sample Estimation: To provide valid inference under even weaker technical conditions, the authors introduce a variant where variable selection is performed on one half of the data and estimation on the other.

Simulation Evidence

The paper’s conclusions are supported by over 22 distinct Monte Carlo simulation designs. These simulations test the method against various challenges, including varying degrees of sparsity, different signal strengths ($R^2$), and complex error structures such as heteroscedasticity and non-normal distributions (e.g., Mixture of Normals).

Across these designs, the Post-Double-Selection estimator consistently matches the performance of an "Oracle" estimator—one that knows the true important variables in advance. Most importantly, it maintains a coverage probability near 95%, whereas single-selection methods fail to provide reliable confidence intervals. This robustness holds true even when the treatment is binary or the errors are non-Gaussian.

Conclusion

The "Double Selection" approach provides a path forward for "honest" inference in high-dimensional settings. By recognizing that variable selection for causal inference is fundamentally different from selection for prediction, researchers can avoid the pitfalls of omitted variable bias. While the method relies on the assumption of "approximate sparsity," its performance in extensive simulations suggests it is a vital tool for ensuring that empirical results are both statistically valid and practically robust.

Summary updated

Abstract

In this supplementary appendix we provide additional results, omitted proofs and extensive simulations that complement the analysis of the main text (arXiv:1201.0224).
BibTeX
@misc{belloni2013supplementary,
  author = {Alexandre Belloni and Victor Chernozhukov and Christian Hansen},
  title = {{Supplementary Appendix for "Inference on Treatment Effects After Selection Amongst High-Dimensional Controls"}},
  year = {2013},
  eprint = {1305.6099},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1305.6099},
}
Figure