Abstract
In modern genetical genomics, researchers must jointly analyze high-dimensional gene expression data and genetic variants to explore their associations with complex traits. A significant challenge arises when both the number of gene expressions ($p$) and genetic variants ($q$) far exceed the sample size ($n$). This paper proposes a Two-Stage Regularization (2SR) framework for variable selection and estimation in high-dimensional sparse instrumental variables (IV) models. By employing sparsity-inducing penalties in both stages, the method identifies optimal instruments and estimates causal effects even when $p, q \gg n$. The approach is shown to be consistent and computationally efficient, outperforming standard penalized methods in both simulations and real-world applications to mouse obesity data.
Introduction: The Challenge of High-Dimensional Confounding
Identifying causal relationships in biological systems is often hindered by endogeneity, where unobserved factors (confounders) influence both the predictors (e.g., gene expression) and the outcome (e.g., a clinical trait). In such cases, standard regression techniques like the Lasso yield biased estimates because they cannot distinguish between correlation and causation.
Instrumental Variables (IV) regression is the classical solution to this problem. An instrument is a variable that affects the outcome only through its effect on the predictors. However, classical IV methods like Two-Stage Least Squares (2SLS) fail in high-dimensional settings where the number of parameters exceeds the number of observations.
As illustrated in Figure 1, the relationship between genetic variants ($z$), gene expressions ($x$), and traits ($y$) is often confounded by unobserved variables ($w$). The 2SR framework is designed to navigate these complex causal structures by selecting the most relevant instruments and predictors from a massive pool of candidates.
The Two-Stage Regularization (2SR) Framework
The proposed 2SR methodology extends the logic of 2SLS into the high-dimensional regime by applying regularization at each step:
- Stage 1 (Instrument Selection): Each endogenous covariate (gene expression) is regressed on the full set of potential instruments (SNPs). Because most SNPs are likely irrelevant to a specific gene, a sparsity-inducing penalty (such as Lasso, SCAD, or MCP) is applied to select the "optimal instruments." This stage produces predicted values for the covariates that are purged of confounding effects.
- Stage 2 (Causal Effect Estimation): The outcome trait is regressed on the predicted covariates from Stage 1. A second round of regularization is applied to select the subset of genes that truly drive the trait.
The authors implement this framework using coordinate descent optimization, ensuring that the method can scale to thousands of variables on standard hardware.
Key Insights and Theoretical Results
A critical contribution of this work is the theoretical proof that the 2SR estimator remains consistent even when the number of covariates and instruments grows exponentially relative to the sample size. The authors also provide a formal proof (Proposition 1) demonstrating that simple penalized least squares (PLS) is fundamentally biased under endogeneity, reinforcing the necessity of the IV approach.
Unlike previous high-dimensional IV methods, 2SR does not require prior knowledge of which instruments correspond to which covariates; the model learns these relationships directly from the data, making it highly adaptable to discovery-based research.
Empirical Performance and Biological Discovery
Simulation Studies
In extensive simulations, the 2SR method consistently outperformed standard PLS. Using the Matthews Correlation Coefficient (MCC) as a metric for variable selection accuracy, 2SR demonstrated a superior ability to identify true causal variables while maintaining a low false-discovery rate, even in the presence of "weak" instruments.
Mouse Obesity Data
The utility of 2SR was demonstrated through an analysis of an F2 mouse intercross dataset ($n=287$) involving $q=1250$ SNPs and $p=2825$ gene expressions. While standard methods flagged hundreds of potentially spurious genes, 2SR identified a sparse set of 17 high-confidence genes. Among these were Igfbp2 and Apoa4, both of which have well-established roles in lipid metabolism and obesity in existing biological literature.
Conclusion
The Two-Stage Regularization framework provides a robust, scalable solution for causal inference in the "Big Data" era of genomics. By accounting for confounding through automated instrument selection, it allows researchers to move beyond simple association studies toward a more mechanistic understanding of complex traits. Future work may extend this framework to more complex estimators like the Generalized Method of Moments (GMM) to further improve efficiency in high-dimensional settings.
Summary updated