Bayesian Information Sharing Between Noise And Regression Models Improves Prediction of Weak Effects

Jussi Gillberg, Pekka Marttinen, Matti Pirinen, Antti J. Kangas, Pasi Soininen, Marjo-Riitta Järvelin, Mika Ala-Korpela, Samuel KaskiarXiv 2013 · arXiv

Abstract

Predicting weak genetic effects in the presence of high-magnitude noise is a central challenge in genomics. This paper introduces a Bayesian multiple-output regression framework that shares structural information between regression weights and noise covariance. By assuming that predictors affect correlated outputs in similar ways, the proposed "Sharing BRRR" model reduces parameter dimensionality and improves prediction accuracy. Empirical results on metabolomics data demonstrate that this approach outperforms state-of-the-art methods like GFlasso while maintaining significantly lower computational costs.

Introduction

In the field of genomics, researchers often struggle to predict "weak effects," where a single genetic variation (e.g., a SNP) accounts for less than 1% of a trait's variance. This task is further complicated by "noise" from latent confounding factors—such as population structure or experimental bias—which can be orders of magnitude larger than the signal. Standard models frequently fail in these "small $n$, large $p$" scenarios due to overfitting and the masking of faint signals by structured noise.

Gillberg et al. (2013) address this by proposing that the regression model and the noise model should share structural information. By imposing mathematical constraints that force the model to ignore noise while capturing faint signals, they significantly improve prediction accuracy in complex biological datasets.

Methodology

The authors propose a model called Sharing BRRR (Bayesian Reduced Rank Regression). This framework integrates several sophisticated Bayesian techniques:

  • Reduced Rank Regression: Instead of estimating a full weight matrix for every predictor-output pair, the model approximates it as a product of two smaller matrices, assuming that many predictors affect the same underlying biological pathways.
  • Infinite Shrinkage Priors: The model uses a prior that automatically penalizes extra components, allowing the data to determine the optimal model complexity (rank) without manual tuning.
  • Information Sharing: The core novelty of the paper is the definition of a joint prior between the regression weights and the noise covariance. The authors assume that if two phenotypes are highly correlated in the noise, the effects of a predictor on those phenotypes are likely to be correlated as well.
Graphical representation of the Bayesian model highlighting the information sharing (red arrows) between noise and regression components.

Experimental Evaluation

The model was evaluated using the NFBC1966 metabolomics dataset from the Northern Finland Birth Cohort 1966. The dataset contains SNP data from 125 metabolism-associated genes and 96 metabolic phenotypes for 4,702 individuals. For comparison, Canonical Correlation Analysis (CCA) was used to reduce the number of SNPs per gene to the 100 most relevant features.

Key Results

The "Sharing BRRR" model consistently achieved lower Mean Squared Error (MSE) than competing methods, including Graph-guided Fused Lasso (GFlasso) and standard Bayesian Linear Models (BLM). The improvement gained from sharing information between the noise and regression components was statistically significant, validating the authors' structural assumptions for biological data.

Comparison of prediction performance (MSE) across different models and sample sizes, showing Sharing BRRR's superior accuracy.

Furthermore, the model proved to be highly efficient. Despite its complexity, it was approximately 10 times faster to train than GFlasso. This efficiency stems from the fully Bayesian treatment, which eliminates the need for computationally expensive cross-validation to tune hyperparameters.

Discussion and Conclusion

A primary insight of this work is that noise structure is informative. Traditionally viewed as a nuisance to be filtered, the correlation structure of noise provides a roadmap for identifying signal locations. By enforcing directional consistency—expecting a SNP to affect correlated metabolites in similar directions—the model achieves better identifiability.

While the current implementation uses some computational approximations to maintain scalability for large samples, future work could explore more rigorous Markov Chain Monte Carlo (MCMC) approaches or further optimize specific model parameters to enhance accuracy. The "sharing" principle introduced here offers a robust strategy for finding subtle signals in high-dimensional, noisy biological data.

Summary updated

Abstract

We consider the prediction of weak effects in a multiple-output regression setup, when covariates are expected to explain a small amount, less than $\approx 1%$, of the variance of the target variables. To facilitate the prediction of the weak effects, we constrain our model structure by introducing a novel Bayesian approach of sharing information between the regression model and the noise model. Further reduction of the effective number of parameters is achieved by introducing an infinite shrinkage prior and group sparsity in the context of the Bayesian reduced rank regression, and using the Bayesian infinite factor model as a flexible low-rank noise model. In our experiments the model incorporating the novelties outperformed alternatives in genomic prediction of rich phenotype data. In particular, the information sharing between the noise and regression models led to significant improvement in prediction accuracy.
BibTeX
@misc{gillberg2013bayesian,
  author = {Jussi Gillberg and Pekka Marttinen and Matti Pirinen and Antti J. Kangas and Pasi Soininen and Marjo-Riitta Järvelin and Mika Ala-Korpela and Samuel Kaski},
  title = {{Bayesian Information Sharing Between Noise And Regression Models Improves Prediction of Weak Effects}},
  year = {2013},
  eprint = {1310.4362},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1310.4362},
}
Figure