Bayesian Structured Sparsity from Gaussian Fields

Barbara E. Engelhardt, Ryan P. AdamsarXiv 2014 · DOI · arXiv

Abstract

Substantial research on structured sparsity has contributed to analysis of many different applications. However, there have been few Bayesian procedures among this work. Here, we develop a Bayesian model for structured sparsity that uses a Gaussian process (GP) to share parameters of the sparsity-inducing prior in proportion to feature similarity as defined by an arbitrary positive definite kernel. For linear regression, this sparsity-inducing prior on regression coefficients is a relaxation of the canonical spike-and-slab prior that flattens the mixture model into a scale mixture of normals. This prior retains the explicit posterior probability on inclusion parameters—now with GP probit prior distributions—but enables tractable computation via elliptical slice sampling for the latent Gaussian field. We motivate development of this prior using the genomic application of association mapping, or identifying genetic variants associated with a continuous trait. Our Bayesian structured sparsity model produced sparse results with substantially improved sensitivity and precision relative to comparable methods. Through simulations, we show that three properties are key to this improvement: i) modeling structure in the covariates, ii) significance testing using the posterior probabilities of inclusion, and iii) model averaging. We present results from applying this model to a large genomic dataset to demonstrate computational tractability.

Introduction: The Challenge of High-Dimensional Structure

In modern data science, particularly genomics, researchers frequently encounter the $p \gg n$ problem: datasets where the number of features (e.g., genetic variants) vastly exceeds the number of samples. To prevent overfitting and ensure interpretability, sparsity-inducing models like the Lasso or Bayesian "spike-and-slab" priors are typically employed to identify a small subset of relevant features.

However, standard tools often treat features as independent entities. In biological systems, features are highly structured. In genomics, this phenomenon is known as Linkage Disequilibrium (LD), where genetic variants located near each other on a chromosome are inherited together and thus exhibit high correlation. Ignoring this structure can lead models to select incorrect variants or fail to detect signals distributed across multiple correlated features.

Methodology: Gaussian Fields for Structured Sparsity

The proposed Bayesian Structured Sparsity (BSS) framework addresses these limitations by incorporating feature similarity directly into the prior. The model utilizes a variation of the spike-and-slab prior where the decision to include a feature is governed by a latent Gaussian field.

Figure 4: Biological inheritance patterns (A) result in the structured correlation matrices, or Linkage Disequilibrium (B), that the BSS model is designed to exploit.

By using a Gaussian Process (GP) with a probit link, the model ensures that highly similar features (as defined by a kernel, such as Pearson correlation) have linked probabilities of inclusion. This "soft sharing" of information allows the model to reflect biological reality more accurately than rigid grouping methods. To maintain computational tractability at scale, the authors employ Elliptical Slice Sampling (ESS), an efficient MCMC technique that avoids the need for complex parameter tuning.

Results and Performance

The model was validated using both synthetic simulations and the HapMap Phase 3 dataset, which includes 40 million SNPs across over 16,000 quantitative traits. The primary goal in these applications is to distinguish between variants that are truly associated with a trait and those that are not.

Figure 1: Comparison of trait distributions across genotypes for a SNP not associated with a trait (a) versus a SNP associated with a trait (b).

In simulations, the structured model (BSSR) consistently outperformed Lasso and independent Bayesian models in precision-recall metrics. When applied to real-world HapMap data, BSSR identified approximately 2.9 million significant genetic associations (eQTLs) at a 5% False Discovery Rate (FDR), a massive increase over the 170,000 found by standard univariate methods. Despite the complexity of GPs, the MCMC approach proved computationally feasible for genome-wide scales through parallel processing.

Discussion and Future Directions

The success of the BSS model highlights the importance of three key Bayesian properties: modeling covariate structure, utilizing Posterior Probabilities of Inclusion (PPI) for significance testing, and model averaging. Unlike frequentist methods that may arbitrarily pick one variant from a correlated group, the Bayesian framework averages over likely models to reduce false positives.

Future work may address "dense-within-groups" sparsity to improve fine-mapping of causal variants and explore epistasis (gene-gene interactions). Additionally, as datasets grow to biobank scales (500,000+ individuals), transitioning from MCMC to Stochastic Variational Inference may be necessary to maintain scalability.

Summary updated

Abstract

Substantial research on structured sparsity has contributed to analysis of many different applications. However, there have been few Bayesian procedures among this work. Here, we develop a Bayesian model for structured sparsity that uses a Gaussian process (GP) to share parameters of the sparsity-inducing prior in proportion to feature similarity as defined by an arbitrary positive definite kernel. For linear regression, this sparsity-inducing prior on regression coefficients is a relaxation of the canonical spike-and-slab prior that flattens the mixture model into a scale mixture of normals. This prior retains the explicit posterior probability on inclusion parameters---now with GP probit prior distributions---but enables tractable computation via elliptical slice sampling for the latent Gaussian field. We motivate development of this prior using the genomic application of association mapping, or identifying genetic variants associated with a continuous trait. Our Bayesian structured sparsity model produced sparse results with substantially improved sensitivity and precision relative to comparable methods. Through simulations, we show that three properties are key to this improvement: i) modeling structure in the covariates, ii) significance testing using the posterior probabilities of inclusion, and iii) model averaging. We present results from applying this model to a large genomic dataset to demonstrate computational tractability.
BibTeX
@misc{engelhardt2014bayesian,
  author = {Barbara E. Engelhardt and Ryan P. Adams},
  title = {{Bayesian Structured Sparsity from Gaussian Fields}},
  year = {2014},
  eprint = {1407.2235},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1407.2235},
}
Figure