Endogeneity in high dimensions

Jianqing Fan, Yuan LiaoThe Annals of Statistics 2014 · DOI · arXiv

Abstract

Most papers on high-dimensional statistics are based on the assumption that none of the regressors are correlated with the regression error, namely, they are exogenous. Yet, endogeneity can arise incidentally from a large pool of regressors in a high-dimensional regression. This causes the inconsistency of the penalized least-squares method and possible false scientific discoveries. A necessary condition for model selection consistency of a general class of penalized regression methods is given, which allows us to prove formally the inconsistency claim. To cope with the incidental endogeneity, we construct a novel penalized focused generalized method of moments (FGMM) criterion function. The FGMM effectively achieves the dimension reduction and applies the instrumental variable methods. We show that it possesses the oracle property even in the presence of endogenous predictors, and that the solution is also near global minimum under the over-identification assumption. Finally, we also show how the semi-parametric efficiency of estimation can be achieved via a two-step approach.

Introduction: The Challenge of Incidental Endogeneity

In modern data science, high-dimensional models where the number of predictors ($p$) significantly exceeds the number of observations ($n$) are common. Standard tools like Lasso or SCAD typically rely on the exogeneity assumption, which posits that predictors are uncorrelated with the model's random noise.

However, in high-dimensional settings, incidental endogeneity is nearly inevitable. With thousands of candidate variables, some will likely correlate with the error term due to measurement errors, omitted variables, or chance. This correlation renders standard penalized least-squares (PLS) methods inconsistent, leading to "false scientific discoveries" where noise-correlated variables are incorrectly identified as significant.

Methodology: Penalized Focused Generalized Method of Moments (FGMM)

To address these challenges, this research introduces the Penalized Focused Generalized Method of Moments (FGMM). This framework integrates Generalized Method of Moments (GMM) with high-dimensional penalty functions through several key innovations:

  1. Instrumental Variables (IV): The method utilizes external "instruments" ($W$) that correlate with predictors but remain uncorrelated with the error term to mitigate endogeneity bias.
  2. The "Focused" Approach: Standard GMM struggles in high dimensions as the number of moment conditions grows, accumulating estimation errors. FGMM uses an indicator function within the loss function to "focus" exclusively on instruments corresponding to non-zero coefficients.
  3. Smoothing and Optimization: To handle the non-smooth nature of the indicator function, a kernel smoothing technique is applied, enabling efficient solution via coordinate descent algorithms.
  4. Two-Step Efficiency: A two-step approach is provided to achieve semi-parametric efficiency, ensuring the estimator is as precise as possible under the model's assumptions.

Simulation Study and Results

The performance of FGMM was evaluated using Monte Carlo simulations across various scenarios, including endogeneity in both signal and noise regressors. The results demonstrate that FGMM possesses the oracle property, performing asymptotically as well as if the true model were known a priori.

Comparison of PLS and FGMM performance across different tuning parameters.

As shown in the table above, FGMM significantly outperforms PLS in terms of model selection accuracy. For instance, at $\lambda=0.05$, PLS identifies an average of 37.68 false positives (FP), whereas FGMM reduces this to a mere 0.08. Furthermore, FGMM maintains a perfect True Positive (TP) rate of 5.0 across all tested $\lambda$ values, while PLS performance degrades as the penalty increases (dropping to 3.63 at $\lambda=1$). The Mean Squared Error (MSE) for both signal ($MSE_S$) and noise ($MSE_N$) components is also substantially lower for FGMM, particularly in the presence of endogenous predictors.

Discussion and Conclusion

The FGMM framework offers a robust solution to the problem of incidental endogeneity in high-dimensional data. By leveraging over-identification and a focused moment selection strategy, it effectively eliminates false positives that plague standard penalized methods.

While the method requires valid instrumental variables—a potential hurdle in some applications—it provides a theoretically sound and empirically superior alternative to PLS. Future research may explore "post-double-selection" methods to further refine inference and address potential model selection imperfections.

Summary updated

Abstract

Most papers on high-dimensional statistics are based on the assumption that none of the regressors are correlated with the regression error, namely, they are exogenous. Yet, endogeneity can arise incidentally from a large pool of regressors in a high-dimensional regression. This causes the inconsistency of the penalized least-squares method and possible false scientific discoveries. A necessary condition for model selection consistency of a general class of penalized regression methods is given, which allows us to prove formally the inconsistency claim. To cope with the incidental endogeneity, we construct a novel penalized focused generalized method of moments (FGMM) criterion function. The FGMM effectively achieves the dimension reduction and applies the instrumental variable methods. We show that it possesses the oracle property even in the presence of endogenous predictors, and that the solution is also near global minimum under the over-identification assumption. Finally, we also show how the semi-parametric efficiency of estimation can be achieved via a two-step approach.
BibTeX
@article{fan2014endogeneity,
  author = {Jianqing Fan and Yuan Liao},
  title = {{Endogeneity in high dimensions}},
  journal = {The Annals of Statistics},
  year = {2014},
  doi = {10.1214/13-aos1202},
}
Figure