Abstract
This paper introduces a unified framework for adaptive inference on both regular (global) and non-regular (local) linear functionals of conditional expectation functions. By utilizing $\ell_1$-regularized Riesz representers within a Neyman orthogonal framework, the authors provide a method for "de-biasing" machine learning estimators. This approach ensures asymptotic uniform validity and "honest" confidence bands, even in high-dimensional settings where the underlying models are complex. The analysis yields a "double sparsity robustness" property, allowing for valid inference even when one of the nuisance components is highly complex.
Introduction
In modern data science, machine learning (ML) models are frequently deployed to predict outcomes in complex environments. However, when the goal shifts from prediction to inference—such as estimating the impact of a policy or a treatment—standard ML "plug-in" estimators often fail. These models are typically tuned to minimize mean squared error, which introduces regularization bias and overfitting that can lead to misleading conclusions.
Double Machine Learning (DML) addresses this by using "orthogonal scores" to cancel out first-order biases. A critical component of these scores is the Riesz representer (RR), a mathematical object that functions as a bias-correction term. Historically, researchers had to derive these RRs analytically for every new problem, a task that is often impossible for complex, "local" parameters (e.g., the effect of a policy on a specific income bracket). This paper automates this process, treating the RR as a nuisance parameter that can be learned directly from data.
Methodology: Automating the Riesz Representer
The core innovation of this work is the automatic estimation of the Riesz representer, extending de-biased ML to a much broader class of parameters.
1. Automatic RR Estimation
Instead of manual derivation, the authors employ a Generalized Dantzig Selector (GDS) with $\ell_1$ regularization. This allows the framework to learn the Riesz representer directly from the data's empirical moment conditions. This automation makes DML a "black-box" tool applicable to a wide range of linear functionals without requiring deep theoretical derivations for each use case.
2. Neyman Orthogonality and Cross-Fitting
The framework constructs a target equation that satisfies Neyman orthogonality, meaning the estimator is insensitive to small errors in the nuisance parameters (the regression function and the RR). To eliminate overfitting bias, the authors utilize cross-fitting: the data is split, the ML models are trained on one portion, and the parameter of interest is estimated on another.
3. Double Sparsity Robustness
A key theoretical contribution is "double sparsity robustness." The authors prove that the method remains valid as long as either the regression function or the Riesz representer is "sparse" (depends on few variables). Even if one component is "completely dense" and highly complex, the estimator can still provide valid results if the other component is sufficiently simple.
Empirical Applications
The authors demonstrate the framework's utility through two real-world economic studies, highlighting its ability to handle both population-wide and subgroup-specific effects.
401(k) Eligibility and Financial Assets
Using the 1991 US Survey of Income and Program Participation (SIPP), the authors estimated the effect of 401(k) eligibility on net financial assets. While they found a global Average Treatment Effect (ATE) of **$7,608** (SE $1,395$), their "local" analysis revealed significant heterogeneity. The effect was statistically insignificant for the second income quintile but very high for the first and fifth quintiles, demonstrating the power of the framework to uncover nuanced subgroup dynamics.
Gasoline Demand Elasticity
In an analysis of the Canadian National Private Vehicle Use Survey, the authors estimated the price elasticity of gasoline demand. While simple OLS models often produce positive elasticities—suggesting demand rises with price, which contradicts economic theory—the de-biased ML approach corrected this bias. They estimated an average price elasticity of -0.28 (SE $0.06$), providing a theoretically sound result that reaches the semi-parametric efficiency bound.
Conclusion and Future Directions
This research provides a robust framework for performing valid inference with machine learning. By automating the estimation of Riesz representers, the authors have created a unified theory that covers both global and local parameters with honest confidence bands.
While the current framework focuses on linear functionals, it sets the stage for future work in nonlinear problems and the application of non-asymptotic bounds to a wider variety of "black-box" learners, such as Random Forests or Deep Neural Networks. This work represents a significant step toward making high-dimensional statistical inference more accessible and reliable for researchers.
Summary updated