Compound random measures and their use in Bayesian nonparametrics

Jim E. Griffin, Fabrizio Leisen

Abstract

This paper introduces Compound Random Measures (CoRM), a new class of dependent random measures, and explores their application as priors in Bayesian nonparametric (BNP) mixture models. By normalizing these measures, we obtain tractable priors that allow for the derivation of key properties, including marginal processes like gamma, $\sigma$-stable, and generalized gamma processes. We characterize dependence through Lévy copulas and correlation functions and propose two efficient posterior inference schemes: a slice sampler and an augmented Pólya urn scheme. The utility of the framework is demonstrated through an analysis of clinical trial data, revealing complex group-specific structures.

Introduction

In Bayesian nonparametrics, modeling unknown distributions often relies on the Dirichlet Process. However, real-world data frequently originate from multiple related sources—such as different cohorts in a clinical trial—where distributions are expected to be similar but not identical. Traditional approaches often fall short: independent priors ignore shared structure, while identical priors assume total homogeneity. Furthermore, existing "Dependent" priors often rely on stick-breaking constructions that impose a strict stochastic ordering, potentially leading to artifacts where the first cluster is consistently the largest across all groups.

To address these limitations, we propose Compound Random Measures (CoRM). This framework provides a mathematically tractable way to define dependent priors that avoid restrictive ordering effects and allow for efficient posterior inference.

Methodology: Compound Random Measures

The core intuition behind CoRMs involves scaling shared random jumps with independent "scores." The construction follows three main steps:

  1. Shared Jumps: A "directing" Lévy process provides a set of shared jump locations and base heights ($J_i$), representing common features across all groups.
  2. Score Distribution: For each group $j$, these shared jumps are multiplied by a random "score" ($m_{j,i}$). A high score emphasizes a feature in a specific group, while a low score suppresses it.
  3. Normalization: To create probability distributions, these measures are normalized (NCoRM).

By varying the score distributions (e.g., using a Gamma distribution), the framework can recover well-known marginal processes such as the Gamma, $\sigma$-stable, and Generalized Gamma processes. The "secret sauce" for tractability is the analytical derivation of the Laplace exponent, which facilitates the development of MCMC algorithms.

Posterior Inference

Efficient computation is a hallmark of this work. We describe two primary strategies:

  • Marginal Method: An augmented Pólya urn scheme that integrates out the random measures.
  • Slice Sampler: A method utilizing latent variables to sample from the posterior without requiring truncation of the infinite series of jumps.

Application: Clinical Trial Analysis

The model was applied to white blood cell count (WBC) response data from two clinical studies: CALGB 8881 and CALGB 9160. These studies involved patients receiving various anticancer drug therapies across nine treatment combinations.

The NCoRM model effectively identified both shared structures and group-specific divergences. While most parameters, such as the timing of the WBC response, were consistent across groups, the model revealed significant differences in specific regression parameters ($\beta_0$ and $\beta_1$).

Posterior samples of regression parameters for CALGB8881 and CALGB9160.

Key Results and Insights

As shown in the figure above, the model discovered that the CALGB 9160 study exhibited a bimodal distribution for the regression parameters, whereas CALGB 8881 was unimodal. This suggests that CALGB 9160 contained two distinct subgroups of patients responding differently to treatment—a nuance that simpler models or those with rigid dependence structures might overlook.

Key advantages of the CoRM framework include:

  • Breaking Stochastic Order: Unlike stick-breaking models, the relative importance of clusters can shift naturally between groups.
  • Unifying Framework: Many previously proposed models are shown to be special cases of CoRM.
  • Rigorous Dependence: The use of Lévy copulas provides a formal mechanism to control information sharing between groups.

Conclusion and Future Work

Compound Random Measures offer a flexible and computationally viable path for dependent BNP modeling. While the current work focuses on independent scores, future research could extend this to "regression-style" dependence, where scores are functions of external covariates like dosage or age. Additionally, further optimizing the computational efficiency of these models remains an active area of interest.

Summary updated

Abstract

A new class of dependent random measures which we call {\it compound random measures} are proposed and the use of normalized versions of these random measures as priors in Bayesian nonparametric mixture models is considered. Their tractability allows the properties of both compound random measures and normalized compound random measures to be derived. In particular, we show how compound random measures can be constructed with gamma, $σ$-stable and generalized gamma process marginals. We also derive several forms of the Laplace exponent and characterize dependence through both the Lévy copula and correlation function. A slice sampler and an augmented Pólya urn scheme sampler are described for posterior inference when a normalized compound random measure is used as the mixing measure in a nonparametric mixture model and a data example is discussed.
BibTeX
@article{griffin2016compound,
  author = {Jim E. Griffin and Fabrizio Leisen},
  title = {{Compound random measures and their use in Bayesian nonparametrics}},
  journal = {Journal of the Royal Statistical Society Series B (Statistical Methodology)},
  year = {2016},
  doi = {10.1111/rssb.12176},
}
Figure