Truncated Inverse-Lévy Measure Representation of the Beta Process

Junyi Zhang, Angelos Dassios, Zhong Chong, Qiufei YaoAISTATS 2025

Introduction

The Beta Process is a fundamental prior in Bayesian nonparametrics, particularly for latent feature modeling where the number of features is not known a priori. However, its infinite-dimensional nature poses significant challenges for practical computation. Traditional truncation methods, such as stick-breaking constructions, often suffer from limited hyperparameter flexibility or high approximation errors due to the inclusion of many negligible "noisy" atoms while potentially omitting significant ones.

The TILe-Rep Framework

In this work, the authors introduce the Truncated Inverse-Lévy measure Representation (TILe-Rep). Unlike standard stick-breaking methods that generate weights in a random order, TILe-Rep generates the $K$ largest atom weights in descending order. This ranked approach ensures that the finite approximation captures the most significant features of the data first, minimizing the posterior approximation error.

Sample average vs. truncation level K for various hyperparameters.

The authors demonstrate that TILe-Rep provides a unified mathematical view, showing that previous constructions by Teh et al. (2007) and Paisley et al. (2010) are special cases or equivalent to this more general representation. By focusing on the $K$ largest jumps of a Poisson process, TILe-Rep achieves the lowest possible posterior approximation error within its class of models.

Inference and Algorithms

To facilitate practical use, the paper develops two main inference schemes:

  • Hamiltonian Monte Carlo (HMC): A gradient-based MCMC sampler. A key innovation here is the use of the Gauss hypergeometric function ($_{2}F_{1}$) to efficiently evaluate the previously intractable "tail" of the Lévy measure, enabling gradient-based updates.
  • Variational Inference (VI): An optimization-based approach that extends mean-field methods to a broader range of hyperparameters, offering a faster alternative to MCMC. In tests, the VI scheme was roughly 2x faster than the Gibbs sampler while maintaining accuracy.

Experimental Evaluation

The effectiveness of TILe-Rep was validated across synthetic and real-world datasets, including MNIST and the Big Five personality traits. In latent feature recovery tasks, the method successfully identifies underlying structures with high accuracy.

Latent feature recovery and posterior atom weights using TILe-Rep.

As shown in Figure 2, both the Gibbs sampler and the VI scheme effectively recover the latent features from observations. The ranked posterior atom weights (Figure 2c) clearly distinguish between significant features and noise, with weights dropping off sharply after the true number of features. In the Big Five personality trait experiment, TILe-Rep produced significantly "cleaner" factor loadings with less background noise compared to older "Weak Approximation" and "Truncated Stick-breaking" methods.

Conclusion

TILe-Rep offers a robust and theoretically grounded framework for approximating the Beta Process. By focusing on ranked atom weights and leveraging efficient computation of special functions, it provides a more accurate and flexible tool for Bayesian nonparametric modeling. Future work includes tightening variational bounds and extending the framework to other classes of Completely Random Measures (CRMs).

Summary updated

BibTeX
@inproceedings{zhang2025truncated,
  author = {Junyi Zhang and Angelos Dassios and Zhong Chong and Qiufei Yao},
  title = {{Truncated Inverse-Lévy Measure Representation of the Beta Process}},
  booktitle = {International Conference on Artificial Intelligence and Statistics},
  year = {2025},
}
Figure