A simple new approach to variable selection in regression, with application to genetic fine-mapping

Gao Wang, Abhishek Sarkar, Peter Carbonetto, Matthew StephensbioRxiv 2018 · DOI

TL;DR

This paper introduces SuSiE, a novel Bayesian approach for variable selection in linear regression that excels at handling highly correlated predictors, a common challenge in fields like genetic fine-mapping. SuSiE's key innovation is the "Sum of Single Effects" model and the "Iterative Bayesian Stepwise Selection" (IBSS) algorithm, which efficiently quantify uncertainty in variable selection by providing "Credible Sets" of likely causal variables.

What Problem Does This Paper Address?

Identifying relevant variables in regression models is crucial across many scientific disciplines. However, existing methods often struggle when predictor variables are highly correlated, making it difficult to distinguish between them and leading to ambiguous conclusions about which variables are truly important. This ambiguity is particularly problematic in applications like genetic fine-mapping, where the goal is to understand the underlying biological mechanisms rather than just build a predictive model. Current methods either arbitrarily select one of several correlated variables or fail to adequately express the uncertainty about which variable is the true signal.

Methods

The core of the paper is the introduction of the "Sum of Single Effects" (SuSiE) model. Instead of assuming a sparse vector of regression coefficients directly, SuSiE models it as a sum of multiple "single-effect" vectors, each with only one non-zero element. This decomposition allows for a more flexible representation of sparsity.

Complementing the model is the Iterative Bayesian Stepwise Selection (IBSS) algorithm. This is a Bayesian analogue of traditional stepwise selection methods but with a crucial difference: instead of picking a single "best" variable at each step, IBSS computes a probability distribution over variables. This distribution captures the uncertainty about which variable is most likely to be the "single effect" at that stage. IBSS is formally justified as a coordinate ascent algorithm that optimizes a variational approximation to the posterior distribution under the SuSiE model. This approach offers computational efficiency and provides "Credible Sets" (CSs) of variables that are likely to contain an effect variable, offering a more nuanced summary of uncertainty than simple posterior inclusion probabilities (PIPs).

Datasets

The authors used real human genotype data from the Genotype-Tissue Expression (GTEx) project for their simulations. This data was used to mimic realistic patterns of correlations among genetic variants (SNPs). They also applied their method to data from Li et al. (2016) for fine-mapping splicing quantitative trait loci (sQTLs).

Key Insights and Ideas

  • Decomposing sparsity: Modeling the sparse regression coefficient vector as a sum of simpler "single-effect" components is a powerful way to handle complex sparsity patterns and maintain computational tractability.
  • Quantifying uncertainty is key: For scientific discovery (like in genetics), simply identifying a "best" set of variables isn't enough. Understanding which variables are uncertainly selected is crucial for drawing reliable conclusions.
  • Bayesian stepwise selection for nuanced inference: IBSS offers a principled way to perform stepwise selection that acknowledges uncertainty, providing distributions over variables rather than just point estimates.
  • Credible Sets as a superior summary: CSs provide a more informative summary of variable selection uncertainty than PIPs, especially in scenarios with highly correlated variables. They aim to be small while still capturing a likely effect variable.
  • Computational efficiency matters: SuSiE achieves state-of-the-art performance in genetic fine-mapping while being significantly faster than existing methods.

Key Results

  • SuSiE, implemented via IBSS, demonstrated strong performance in numerical experiments, particularly in challenging genetic fine-mapping scenarios with highly correlated variables.
  • The method produced results comparable to or better than existing Bayesian variable selection methods (CAVIAR, FINEMAP, DAP-G) in terms of accuracy (e.g., power vs. False Discovery Rate).
  • SuSiE was significantly faster than other methods, achieving speedups of up to 4,000 times compared to CAVIAR.
  • The Credible Sets produced by SuSiE showed good coverage and were generally smaller and purer than those from other methods.
  • The approach was successfully applied to fine-mapping splicing QTLs, identifying primary and secondary signals of association.

Limitations and Future Work

  • Local optima in IBSS: The authors acknowledge that the IBSS algorithm, while generally effective, can sometimes converge to poor local optima, particularly in change point detection problems or when there are many effect variables. They suggest that better initialization strategies or more advanced optimization algorithms could address this.
  • Generalizability to GLMs: While the current work focuses on linear regression, extending SuSiE to generalized linear models (GLMs), such as logistic regression, would be a valuable next step, especially for modeling binary outcomes.
  • Handling summary statistics: Adapting SuSiE to work with summary statistics (rather than full data) would broaden its applicability, especially for large-scale genetic studies.
  • Multivariate outcomes: Extending the framework to simultaneously select variables for multiple outcomes (multivariate regression and multi-task learning) is a promising direction for enhancing power and precision.

Summary updated

Abstract

We introduce a simple new approach to variable selection in linear regression, with a particular focus on quantifying uncertainty in which variables should be selected . The approach is based on a new model — the “Sum of Single Effects” ( SuSiE ) model — which comes from writing the sparse vector of regression coefficients as a sum of “single-effect” vectors, each with one non-zero element. We also introduce a corresponding new fitting procedure — Iterative Bayesian Stepwise Selection (IBSS) — which is a Bayesian analogue of stepwise selection methods. IBSS shares the computational simplicity and speed of traditional stepwise methods, but instead of selecting a single variable at each step, IBSS computes a distribution on variables that captures uncertainty in which variable to select. We provide a formal justification of this intuitive algorithm by showing that it optimizes a variational approximation to the posterior distribution under the SuSiE model. Further, this approximate posterior distribution naturally yields convenient novel summaries of uncertainty in variable selection, providing a Credible Set of variables for each selection. Our methods are particularly well-suited to settings where variables are highly correlated and detectable effects are sparse, both of which are characteristics of genetic fine-mapping applications. We demonstrate through numerical experiments that our methods outper-form existing methods for this task, and illustrate their application to fine-mapping genetic variants influencing alternative splicing in human cell-lines. We also discuss the potential and challenges for applying these methods to generic variable selection problems.
BibTeX
@misc{wang2018simple,
  author = {Gao Wang and Abhishek Sarkar and Peter Carbonetto and Matthew Stephens},
  title = {{A simple new approach to variable selection in regression, with application to genetic fine-mapping}},
  year = {2018},
  doi = {10.1101/501114},
  howpublished = {bioRxiv},
}
Figure