FBMS: Flexible Bayesian Model Selection and Model Averaging

Jon Lachmann, Aliaksandr HubinOpenalex Unknown 2023 · DOI · OpenAlex

Abstract

Implements the Mode Jumping Markov Chain Monte Carlo algorithm described in <a href="https://doi.org/10.1016%2Fj.csda.2018.05.020" target="_top">doi:10.1016/j.csda.2018.05.020</a> and its Genetically Modified counterpart described in <a href="https://doi.org/10.1613%2Fjair.1.13047" target="_top">doi:10.1016/j.jair.1.13047</a> as well as the sub-sampling versions described in <a href="https://doi.org/10.1016%2Fj.ijar.2022.08.018" target="_top">doi:10.1016/j.ijar.2022.08.018</a> for flexible Bayesian model selection and model averaging.

Introduction

In modern statistical modeling, researchers frequently encounter two significant hurdles: model uncertainty and inherent nonlinearity. When faced with numerous potential predictors, identifying the single "correct" subset is often impossible, and assuming perfectly linear relationships can lead to poor predictive performance. The FBMS (Flexible Bayesian Model Selection) package provides a robust R-based toolkit designed to navigate these complexities using advanced Markov Chain Monte Carlo (MCMC) techniques.

Algorithmic Innovations

Traditional MCMC methods often struggle with "local optima," becoming trapped in regions of the model space that are high-performing but not globally optimal. FBMS introduces three primary innovations to overcome this:

  1. Mode Jumping MCMC (MJMCMC): This algorithm utilizes large, discrete "jumps" to traverse different high-probability regions of the model space, ensuring a more comprehensive exploration than local-step methods.
  2. Genetically Modified MJMCMC (GMJMCMC): By incorporating evolutionary principles, GMJMCMC treats nonlinear transformations as "features" that can evolve, mutate, and cross-over. This allows the discovery of complex structures like log(x) * sin(y) automatically.
  3. Sub-sampling Versions: To address "Big Data" constraints, FBMS includes versions that use stochastic approximations of likelihood based on data sub-samples, significantly reducing computational overhead for massive datasets.

Nonlinear Transformations and Feature Engineering

A key strength of FBMS is its ability to handle a vast library of nonlinear transformations out of the box. These include sigmoids, ReLUs, and various polynomial terms, which are essential for capturing real-world dynamics that linear models miss.

The arcsinh transformation is one of many nonlinear functions supported by FBMS for automated feature engineering.

Bayesian Model Averaging (BMA)

Rather than relying on a single "best" model, FBMS promotes Bayesian Model Averaging. By weighting predictions across an ensemble of likely models, BMA produces more robust results and reduces the risk of overfitting. The package provides specific utilities to manage these aggregated results efficiently.

The aggr function serves as a generic interface for accessing and managing aggregated predictions within the FBMS framework.

Applications and Scalability

The package demonstrates its versatility through several built-in datasets, ranging from physical measurements (abalone) and medical diagnostics (breastcancer) to high-dimensional genomic data (SangerData2). To handle the computational intensity of these searches, FBMS provides parallelized functions (mjmcmc.parallel, gmjmcmc.parallel) to distribute workloads across multiple CPU cores.

Conclusion

FBMS bridges the gap between traditional local-search MCMC and global optimization. While searching nonlinear spaces remains computationally demanding, the package's ability to automate feature engineering and provide robust averaged predictions makes it a powerful tool for modern statistical analysis. Future developments focus on improving sub-sampling efficiency and expanding the library of supported likelihood functions.

Summary updated

Abstract

Implements the Mode Jumping Markov Chain Monte Carlo algorithm described in &lt;<a href="https://doi.org/10.1016%2Fj.csda.2018.05.020" target="_top">doi:10.1016/j.csda.2018.05.020</a>&gt; and its Genetically Modified counterpart described in &lt;<a href="https://doi.org/10.1613%2Fjair.1.13047" target="_top">doi:10.1613/jair.1.13047</a>&gt; as well as the sub-sampling versions described in &lt;<a href="https://doi.org/10.1016%2Fj.ijar.2022.08.018" target="_top">doi:10.1016/j.ijar.2022.08.018</a>&gt; for flexible Bayesian model selection and model averaging.
BibTeX
@inproceedings{lachmann2023fbms,
  author = {Jon Lachmann and Aliaksandr Hubin},
  title = {{FBMS: Flexible Bayesian Model Selection and Model Averaging}},
  booktitle = {Openalex Unknown},
  year = {2023},
  doi = {10.32614/cran.package.fbms},
}
Figure