Abstract
As high-dimensional empirical applications become increasingly prevalent across research disciplines, the need for valid simultaneous inference has grown. High-dimensional settings—where the number of potential covariates $p$ may exceed the number of observations $n$—frequently arise in economic studies involving rich datasets or the analysis of treatment heterogeneities. Traditional methods often fail in these contexts due to post-selection bias and the inflation of false positives in multiple testing. This paper reviews classical and modern methods for simultaneous inference and demonstrates their implementation via the R package hdm. By combining "double-selection" LASSO with the Romano-Wolf stepdown procedure and multiplier bootstrap techniques, the package provides a robust framework for performing valid post-selection inference and constructing simultaneous confidence intervals.
Introduction
In modern data science and econometrics, researchers are increasingly confronted with "rich" datasets where the number of variables is large relative to the sample size. Whether analyzing the drivers of economic growth or the nuances of the gender wage gap, the availability of numerous potential covariates and their interactions creates a high-dimensional environment. While machine learning tools like LASSO (Least Absolute Shrinkage and Selection Operator) are excellent for prediction and variable selection, they are not inherently designed for hypothesis testing. Using standard inferential tools after variable selection often leads to biased estimates and spurious discoveries.
This paper explores a practical framework for valid statistical inference in these settings, utilizing the hdm (High-Dimensional Metrics) package for R. The goal is to bridge the gap between the predictive power of machine learning and the rigorous requirements of classical statistical inference.
The Dual Challenges of High-Dimensional Inference
1. The High-Dimensional Curse and Post-Selection Bias
Traditional regression methods, such as Ordinary Least Squares (OLS), become unstable or mathematically impossible when $p \approx n$ or $p > n$. While LASSO addresses this by shrinking coefficients and performing variable selection, it introduces a "selection bias." If a researcher selects variables based on their predictive power and then runs a standard regression on those selected variables, the resulting p-values and confidence intervals are invalid. This phenomenon, known as post-selection inference, often leads to overconfident results and omitted variable bias if the selection process misses variables correlated with both the treatment and the outcome.
2. The Multiple Testing Problem
In high-dimensional settings, researchers often test many hypotheses simultaneously. If 100 variables are tested at a 5% significance level, one expects five "significant" results purely by chance. Without proper adjustment, the Family-Wise Error Rate (FWER)—the probability of making at least one Type I error—approaches 1.0, making false discoveries virtually certain.
Methodological Framework
The hdm package implements three key technical components to address these challenges:
Double-Selection LASSO
To mitigate the bias introduced by standard variable selection, the "double-selection" method performs LASSO twice:
- Step 1: Select variables that predict the outcome ($y$).
- Step 2: Select variables that predict the "target" variable of interest ($d$).
By including the union of variables from both steps in a final regression, the researcher "protects" the inference against omitted variable bias. This ensures that even if a variable has a small effect on the outcome but is highly correlated with the target variable, it is still included in the model, preserving the validity of the target variable's coefficient estimate.
Multiplier Bootstrap
Traditional bootstrapping, which involves resampling the entire dataset, is computationally prohibitive in high-dimensional contexts. The hdm package utilizes the multiplier bootstrap, a faster alternative that perturbs the influence functions of the estimators with random noise. This method allows for the efficient construction of simultaneous confidence regions and the calculation of adjusted p-values without the need for exhaustive resampling.
Romano-Wolf Stepdown Procedure
Unlike the Bonferroni correction, which is often overly conservative because it assumes independence between tests, the Romano-Wolf procedure accounts for the dependence structure between test statistics. By using the bootstrap to capture how variables correlate, it can reject more false null hypotheses (providing higher statistical power) while still strictly controlling the FWER.
Empirical Evaluation
Synthetic Simulation
The authors tested the framework using a controlled simulation with $K=60$ regressors and sample sizes of $n=200$ and $n=500$. The data was designed to be "sparse" (only 12 variables were truly active) with high correlation ($\rho=0.9$) between predictors. The results demonstrated that the Romano-Wolf procedure consistently achieved the highest number of correct rejections among all methods that successfully controlled the FWER. In contrast, "naive" approaches that ignored multiple testing adjustments resulted in an empirical FWER of nearly 1.0.
Case Study: The Gender Wage Gap
Using data from the 2016 American Community Survey (ACS) involving 70,473 full-time workers, the authors analyzed gender wage gap heterogeneity. The model included 123 regressors, including complex interactions between gender, industry, occupation, and education.
A "naive" analysis suggested that 25 different gender-interaction terms were statistically significant. However, after applying the Romano-Wolf correction via the hdm package, only 11 remained significant. This suggests that many perceived "drivers" of the wage gap in simpler models may actually be statistical noise, highlighting the importance of rigorous simultaneous inference.
Conclusion and Limitations
The hdm package provides a powerful toolkit for researchers working with high-dimensional data, allowing for valid inference that accounts for both selection bias and multiple testing. However, these methods rely on the assumption of "approximate sparsity"—the idea that a relatively small number of variables carry the majority of the signal. If the true underlying model is "dense," where many variables have small, equal effects, LASSO-based inference may perform poorly. Additionally, while the multiplier bootstrap is efficient, scaling these methods to thousands of simultaneous target variables remains a computational challenge. Future work continues to explore extending these valid inference tools to non-linear and non-parametric models.
Summary updated