TL;DR
This paper introduces SuSiE, a novel Bayesian approach for variable selection in linear regression that excels at handling highly correlated predictors, a common challenge in fields like genetic fine-mapping. SuSiE's key innovation is the "Sum of Single Effects" model and the "Iterative Bayesian Stepwise Selection" (IBSS) algorithm, which efficiently quantify uncertainty in variable selection by providing "Credible Sets" of likely causal variables.
What Problem Does This Paper Address?
Identifying relevant variables in regression models is crucial across many scientific disciplines. However, existing methods often struggle when predictor variables are highly correlated, making it difficult to distinguish between them and leading to ambiguous conclusions about which variables are truly important. This ambiguity is particularly problematic in applications like genetic fine-mapping, where the goal is to understand the underlying biological mechanisms rather than just build a predictive model. Current methods either arbitrarily select one of several correlated variables or fail to adequately express the uncertainty about which variable is the true signal.
Methods
The core of the paper is the introduction of the "Sum of Single Effects" (SuSiE) model. Instead of assuming a sparse vector of regression coefficients directly, SuSiE models it as a sum of multiple "single-effect" vectors, each with only one non-zero element. This decomposition allows for a more flexible representation of sparsity.
Complementing the model is the Iterative Bayesian Stepwise Selection (IBSS) algorithm. This is a Bayesian analogue of traditional stepwise selection methods but with a crucial difference: instead of picking a single "best" variable at each step, IBSS computes a probability distribution over variables. This distribution captures the uncertainty about which variable is most likely to be the "single effect" at that stage. IBSS is formally justified as a coordinate ascent algorithm that optimizes a variational approximation to the posterior distribution under the SuSiE model. This approach offers computational efficiency and provides "Credible Sets" (CSs) of variables that are likely to contain an effect variable, offering a more nuanced summary of uncertainty than simple posterior inclusion probabilities (PIPs).
Datasets
The authors used real human genotype data from the Genotype-Tissue Expression (GTEx) project for their simulations. This data was used to mimic realistic patterns of correlations among genetic variants (SNPs). They also applied their method to data from Li et al. (2016) for fine-mapping splicing quantitative trait loci (sQTLs).
Key Insights and Ideas
- Decomposing sparsity: Modeling the sparse regression coefficient vector as a sum of simpler "single-effect" components is a powerful way to handle complex sparsity patterns and maintain computational tractability.
- Quantifying uncertainty is key: For scientific discovery (like in genetics), simply identifying a "best" set of variables isn't enough. Understanding which variables are uncertainly selected is crucial for drawing reliable conclusions.
- Bayesian stepwise selection for nuanced inference: IBSS offers a principled way to perform stepwise selection that acknowledges uncertainty, providing distributions over variables rather than just point estimates.
- Credible Sets as a superior summary: CSs provide a more informative summary of variable selection uncertainty than PIPs, especially in scenarios with highly correlated variables. They aim to be small while still capturing a likely effect variable.
- Computational efficiency matters: SuSiE achieves state-of-the-art performance in genetic fine-mapping while being significantly faster than existing methods.
Key Results
- SuSiE, implemented via IBSS, demonstrated strong performance in numerical experiments, particularly in challenging genetic fine-mapping scenarios with highly correlated variables.
- The method produced results comparable to or better than existing Bayesian variable selection methods (CAVIAR, FINEMAP, DAP-G) in terms of accuracy (e.g., power vs. False Discovery Rate).
- SuSiE was significantly faster than other methods, achieving speedups of up to 4,000 times compared to CAVIAR.
- The Credible Sets produced by SuSiE showed good coverage and were generally smaller and purer than those from other methods.
- The approach was successfully applied to fine-mapping splicing QTLs, identifying primary and secondary signals of association.
Limitations and Future Work
- Local optima in IBSS: The authors acknowledge that the IBSS algorithm, while generally effective, can sometimes converge to poor local optima, particularly in change point detection problems or when there are many effect variables. They suggest that better initialization strategies or more advanced optimization algorithms could address this.
- Generalizability to GLMs: While the current work focuses on linear regression, extending SuSiE to generalized linear models (GLMs), such as logistic regression, would be a valuable next step, especially for modeling binary outcomes.
- Handling summary statistics: Adapting SuSiE to work with summary statistics (rather than full data) would broaden its applicability, especially for large-scale genetic studies.
- Multivariate outcomes: Extending the framework to simultaneously select variables for multiple outcomes (multivariate regression and multi-task learning) is a promising direction for enhancing power and precision.
Summary updated