Introduction
In Bayesian statistics, conjugacy is a prized property that ensures the posterior distribution remains within the same functional family as the prior, facilitating tractable updates as new data arrives. For finite-dimensional data, the theory of exponential families provides a unified framework for identifying conjugate pairs. However, in Bayesian Nonparametrics (BNP)—which operates in infinite-dimensional spaces with objects like Dirichlet or Beta processes—this unification has historically been absent. Researchers have typically had to derive conjugacy for each model individually using complex, case-specific proofs.
This paper addresses this gap by providing a general mathematical "recipe" for constructing conjugate BNP models. By bridging the gap between finite-dimensional exponential families and infinite-dimensional Lévy measures, the authors offer a systematic way to derive nonparametric priors from standard distributions.
The Sufficient Statistic Framework
The core innovation of this work is the use of sufficient statistics to define the density of a Lévy measure. In standard statistics, a sufficient statistic captures all relevant information from data to estimate a parameter. The authors generalize a construction by Nils Lid Hjort (1990) to show that if a distribution belongs to an $m$-dimensional exponential family, its sufficient statistics can govern the infinitesimal increments of a Lévy process.
Their primary result, Theorem 1, formalizes the construction of a Lévy process where increments are distributed according to a chosen exponential family. By employing a change-of-variables technique, they ensure that the resulting Completely Random Measure (CRM)—the infinite-dimensional analogue of a random variable—maintains the distributional properties necessary for conjugacy. The paper also provides a sampling algorithm (Algorithm 1) to facilitate practical implementation of these models.
Unifying and Extending BNP Models
A major strength of this framework is its ability to unify existing models and generate new ones. The authors demonstrate that classic BNP priors, such as the Beta process and Gamma process, are specific instances of their general construction. By identifying the appropriate sufficient statistics and natural parameters, these models emerge naturally from the framework.
Beyond unification, the authors introduce a new CRM based on the Pareto distribution. This "Pareto process" provides a heavy-tailed alternative for researchers, demonstrating the framework's utility in expanding the BNP toolkit. As this is a theoretical work, evaluation is conducted through mathematical proof rather than empirical datasets, demonstrating that existing models fit this new paradigm.
The figure above illustrates the additive nature of the constructed measures, showing how individual components ($\nu_1, \nu_2, \nu_k$) aggregate into a final measure $\nu$. This property is central to the behavior of Completely Random Measures in this framework.
Conclusion and Future Directions
While this work provides a powerful foundation for constructing BNP priors, it represents the first stage of a broader research program. Future work is needed to provide a general proof for the posterior update step to fully complete the theory of conjugacy. Additionally, the current framework is restricted to positive-valued exponential families and requires certain regularity conditions, such as the differentiability of the inverse sufficient statistic.
Despite these limitations, the sufficient statistics construction offers a significant step toward a more principled and automated approach to Bayesian nonparametric modeling, moving the field away from one-off proofs toward a unified theoretical landscape.
Summary updated