Pseudo-Marginal Slice Sampling

Iain Murray, Matthew M. GrahamAISTATS 2016

Abstract

Pseudo-marginal MCMC is a powerful technique for sampling from distributions where only an unbiased estimate of the density is available. However, high variance in these estimates often causes the Markov chain to "stick" at points where the density is overestimated. We present the Auxiliary Pseudo-Marginal (APM) framework, which treats the randomness of the estimator as auxiliary variables. By alternating updates between the model parameters and this randomness, APM enables the use of robust methods like slice sampling. This approach significantly improves mixing, eliminates sticking behavior, and restores standard tuning heuristics for complex Bayesian inference.

Introduction

In Bayesian statistics, sampling from a posterior distribution $\pi(\theta)$ often requires evaluating a target density $f(\theta)$. For many sophisticated models—such as Gaussian processes or models with intractable normalizing constants—calculating $f(\theta)$ is computationally impossible. Pseudo-marginal MCMC (PM-MCMC) addresses this by substituting $f(\theta)$ with an unbiased estimate $\hat{f}(\theta)$.

While theoretically sound, PM-MCMC frequently fails in practice due to sticking. If the estimator produces a significant overestimate of the density at a specific point, the Metropolis-Hastings acceptance probability for any subsequent move becomes extremely low. The chain effectively becomes trapped, sometimes for thousands of iterations, leading to poor mixing and unreliable inference.

The Auxiliary Pseudo-Marginal (APM) Framework

The authors propose a shift in perspective: the randomness used to generate the estimate is not just noise, but a set of auxiliary variables ($u$). By defining the estimator as a deterministic function $\hat{f}(\theta; u)$, the Markov chain state is expanded to include both the parameters $\theta$ and the "seed" $u$.

The APM framework employs an alternating update scheme:

  1. Update $u$: The auxiliary variables are updated while the parameters $\theta$ remain fixed.
  2. Update $\theta$: The parameters are updated while the randomness $u$ is "clamped."
Figure 1: Comparison of PM-MH and APM tuning. (a-b) show how APM restores predictable acceptance rates vs step size. (d) shows significantly improved cost-scaled autocorrelation for APM variants.

By clamping $u$ during the parameter update, the target density becomes deterministic. This "unlocks" advanced MCMC tools like Slice Sampling, which previously could not be used in pseudo-marginal settings. Slice sampling is particularly advantageous because it is adaptive; it automatically adjusts its step size to ensure the parameters $\theta$ almost always move, even when the underlying estimator is noisy.

Experimental Results

The APM framework was evaluated on a variety of tasks, including a 5D Gaussian toy model, the Ising model, and Gaussian Process (GP) classification using UCI datasets.

Mixing and Efficiency

The most striking result is the elimination of sticking. Trace plots from the Ising model experiments clearly show standard PM-MH plateauing, while APM methods maintain fluid mixing.

Figure 2: Performance on the Ising model. (a-d) Trace plots showing PM-MH sticking vs. the consistent mixing of APM. (e-f) Cost-scaled autocorrelations demonstrating the superior efficiency of APM SS+SS.

In GP classification on the Breast Cancer dataset, the APM SS+SS method more than doubled the Effective Sample Size (ESS) compared to standard pseudo-marginal MH, even after accounting for the increased computational cost of the two-step update. Furthermore, APM restores the ability to use standard tuning heuristics, such as aiming for specific acceptance rates, which typically fail in the presence of estimator noise.

Conclusion

The Auxiliary Pseudo-Marginal framework offers a robust solution to the sticking problem in pseudo-marginal MCMC. By treating estimator randomness as a latent state, it allows researchers to leverage powerful, adaptive sampling algorithms like slice sampling. The result is a more reliable, easier-to-tune, and significantly more efficient tool for black-box Bayesian inference in complex models.

Summary updated

BibTeX
@inproceedings{murray2016pseudo,
  author = {Iain Murray and Matthew M. Graham},
  title = {{Pseudo-Marginal Slice Sampling}},
  booktitle = {International Conference on Artificial Intelligence and Statistics},
  year = {2016},
}
Figure