Collapsed Variational Inference for Nonparametric Bayesian Group Factor Analysis

Sikun Yang, Heinz Koeppl

Introduction

In modern data science, researchers frequently encounter multiple related datasets—such as gene expression and DNA copy numbers in systems biology—that share underlying structures. Group Factor Analysis (GFA) is a powerful framework for identifying these shared and group-specific latent factors. However, existing inference methods for GFA face a significant bottleneck: Markov Chain Monte Carlo (MCMC) is accurate but computationally prohibitive for large datasets, while standard Variational Inference (VI) is fast but often fails to capture the tight dependencies between latent variables due to the restrictive mean-field assumption.

Methodology: Nonparametric Bayesian GFA

This paper introduces Nonparametric Bayesian Group Factor Analysis (NGFA), which leverages a hierarchical Beta Bernoulli Process to automatically infer the optimal number of latent factors ($K$). To induce structured sparsity, the model utilizes a Bernoulli-Gaussian construction ($G = Z \odot W$), effectively distinguishing between factors that are shared across groups and those unique to a single dataset.

The core innovation is the use of Collapsed Variational Inference (CVI). By analytically marginalizing (collapsing) the group-specific beta process parameters before applying the mean-field approximation, the authors transform the latent space. In this collapsed space, variables interact through a "field" of sufficient statistics, making them weakly dependent. This allows the mean-field assumption to hold more accurately, bridging the gap between the speed of VI and the accuracy of MCMC.

Experimental Evaluation and Results

The NGFA model was rigorously tested against state-of-the-art methods across synthetic and real-world datasets.

Synthetic Data: Sparsity Recovery

On simulated data with varying sample sizes ($N=20$ to $80$), the model's ability to recover sparse and dense loading patterns was evaluated using the Sparse Stability Index (SSI) and Dense Stability Index (DSI).

Stability indices for sparsity recovery on synthetic data across varying sample sizes (N).

As shown in the figure above, NGFA consistently achieved higher SSI and lower DSI compared to methods like BASS and sGFA, demonstrating superior robustness, particularly in small-sample regimes.

Cancer Genomics: Gene Prioritization

The model was applied to the Hyman and Pollack datasets to integrate Gene Expression (GE) and Copy Number Variation (CNV) data. The goal was to prioritize cancer-related genes by identifying shared signals across these data views.

AUC performance for gene prioritization on the Hyman and Pollack datasets.

NGFA achieved the highest Area Under the ROC Curve (AUC) among 14 competing methods, including BIBFA and PMA. This result underscores the model's ability to extract biologically meaningful shared structures that other methods might miss.

Neuroimaging: fMRI Decoding

In a visual reconstruction task using fMRI data, NGFA outperformed Bayesian Canonical Correlation Analysis, achieving a lower Mean Squared Error (0.224 vs. 0.251) in reconstructing geometric shapes and letters from brain activity.

Conclusion

By combining nonparametric Bayesian modeling with collapsed variational inference, the authors provide a scalable and accurate solution for Group Factor Analysis. NGFA effectively handles the "speed vs. accuracy" trade-off, offering a robust tool for multi-view data integration in complex fields like genomics and neuroscience. Future work may extend this framework to dynamic time-series data and hierarchical factor structures.

Summary updated

BibTeX
@inproceedings{yang2018collapsed,
  author = {Sikun Yang and Heinz Koeppl},
  title = {{Collapsed Variational Inference for Nonparametric Bayesian Group Factor Analysis}},
  booktitle = {ICDM},
  year = {2018},
  doi = {10.1109/icdm.2018.00084},
}
Figure