Regularization Methods for High-Dimensional Instrumental Variables Regression With an Application to Genetical Genomics

Wei Lin, Rui Feng, Hongzhe LiJournal of the American Statistical Association 2014 · DOI · arXiv

Abstract

In modern genetical genomics, researchers must jointly analyze high-dimensional gene expression data and genetic variants to explore their associations with complex traits. A significant challenge arises when both the number of gene expressions ($p$) and genetic variants ($q$) far exceed the sample size ($n$). This paper proposes a Two-Stage Regularization (2SR) framework for variable selection and estimation in high-dimensional sparse instrumental variables (IV) models. By employing sparsity-inducing penalties in both stages, the method identifies optimal instruments and estimates causal effects even when $p, q \gg n$. The approach is shown to be consistent and computationally efficient, outperforming standard penalized methods in both simulations and real-world applications to mouse obesity data.

Introduction: The Challenge of High-Dimensional Confounding

Identifying causal relationships in biological systems is often hindered by endogeneity, where unobserved factors (confounders) influence both the predictors (e.g., gene expression) and the outcome (e.g., a clinical trait). In such cases, standard regression techniques like the Lasso yield biased estimates because they cannot distinguish between correlation and causation.

Instrumental Variables (IV) regression is the classical solution to this problem. An instrument is a variable that affects the outcome only through its effect on the predictors. However, classical IV methods like Two-Stage Least Squares (2SLS) fail in high-dimensional settings where the number of parameters exceeds the number of observations.

Figure 1: Causal diagrams illustrating the relationship between instruments (z), covariates (x), outcomes (y), and unobserved confounders (w). (a) shows a basic IV setup, while (b) and (c) represent more complex network structures.

As illustrated in Figure 1, the relationship between genetic variants ($z$), gene expressions ($x$), and traits ($y$) is often confounded by unobserved variables ($w$). The 2SR framework is designed to navigate these complex causal structures by selecting the most relevant instruments and predictors from a massive pool of candidates.

The Two-Stage Regularization (2SR) Framework

The proposed 2SR methodology extends the logic of 2SLS into the high-dimensional regime by applying regularization at each step:

  1. Stage 1 (Instrument Selection): Each endogenous covariate (gene expression) is regressed on the full set of potential instruments (SNPs). Because most SNPs are likely irrelevant to a specific gene, a sparsity-inducing penalty (such as Lasso, SCAD, or MCP) is applied to select the "optimal instruments." This stage produces predicted values for the covariates that are purged of confounding effects.
  2. Stage 2 (Causal Effect Estimation): The outcome trait is regressed on the predicted covariates from Stage 1. A second round of regularization is applied to select the subset of genes that truly drive the trait.

The authors implement this framework using coordinate descent optimization, ensuring that the method can scale to thousands of variables on standard hardware.

Key Insights and Theoretical Results

A critical contribution of this work is the theoretical proof that the 2SR estimator remains consistent even when the number of covariates and instruments grows exponentially relative to the sample size. The authors also provide a formal proof (Proposition 1) demonstrating that simple penalized least squares (PLS) is fundamentally biased under endogeneity, reinforcing the necessity of the IV approach.

Unlike previous high-dimensional IV methods, 2SR does not require prior knowledge of which instruments correspond to which covariates; the model learns these relationships directly from the data, making it highly adaptable to discovery-based research.

Empirical Performance and Biological Discovery

Simulation Studies

In extensive simulations, the 2SR method consistently outperformed standard PLS. Using the Matthews Correlation Coefficient (MCC) as a metric for variable selection accuracy, 2SR demonstrated a superior ability to identify true causal variables while maintaining a low false-discovery rate, even in the presence of "weak" instruments.

Mouse Obesity Data

The utility of 2SR was demonstrated through an analysis of an F2 mouse intercross dataset ($n=287$) involving $q=1250$ SNPs and $p=2825$ gene expressions. While standard methods flagged hundreds of potentially spurious genes, 2SR identified a sparse set of 17 high-confidence genes. Among these were Igfbp2 and Apoa4, both of which have well-established roles in lipid metabolism and obesity in existing biological literature.

Conclusion

The Two-Stage Regularization framework provides a robust, scalable solution for causal inference in the "Big Data" era of genomics. By accounting for confounding through automated instrument selection, it allows researchers to move beyond simple association studies toward a more mechanistic understanding of complex traits. Future work may extend this framework to more complex estimators like the Generalized Method of Moments (GMM) to further improve efficiency in high-dimensional settings.

Summary updated

Abstract

In genetical genomics studies, it is important to jointly analyze gene expression data and genetic variants in exploring their associations with complex traits, where the dimensionality of gene expressions and genetic variants can both be much larger than the sample size. Motivated by such modern applications, we consider the problem of variable selection and estimation in high-dimensional sparse instrumental variables models. To overcome the difficulty of high dimensionality and unknown optimal instruments, we propose a two-stage regularization framework for identifying and estimating important covariate effects while selecting and estimating optimal instruments. The methodology extends the classical two-stage least squares estimator to high dimensions by exploiting sparsity using sparsity-inducing penalty functions in both stages. The resulting procedure is efficiently implemented by coordinate descent optimization. For the representative <i>L</i><sub>1</sub> regularization and a class of concave regularization methods, we establish estimation, prediction, and model selection properties of the two-stage regularized estimators in the high-dimensional setting where the dimensionality of co-variates and instruments are both allowed to grow exponentially with the sample size. The practical performance of the proposed method is evaluated by simulation studies and its usefulness is illustrated by an analysis of mouse obesity data. Supplementary materials for this article are available online.
BibTeX
@article{lin2014regularization,
  author = {Wei Lin and Rui Feng and Hongzhe Li},
  title = {{Regularization Methods for High-Dimensional Instrumental Variables Regression With an Application to Genetical Genomics}},
  journal = {Journal of the American Statistical Association},
  year = {2014},
  doi = {10.1080/01621459.2014.908125},
}
Figure