Adaptive Bernstein–von Mises theorems in Gaussian white noise

Kolyan RayThe Annals of Statistics 2017 · DOI · arXiv

Abstract

We investigate Bernstein–von Mises theorems for adaptive nonparametric Bayesian procedures in the canonical Gaussian white noise model. We consider both a Hilbert space and multiscale setting with applications in $L^{2}$ and $L^{\infty}$, respectively. This provides a theoretical justification for plug-in procedures, for example the use of certain credible sets for sufficiently smooth linear functionals. We use this general approach to construct optimal frequentist confidence sets based on the posterior distribution. We also provide simulations to numerically illustrate our approach and obtain a visual representation of the geometries involved.

Introduction

In modern statistics, quantifying uncertainty is as critical as the estimation itself. Bayesian methods provide "credible sets" based on the posterior distribution, while frequentist methods rely on "confidence sets" with guaranteed coverage under repeated sampling. The Bernstein–von Mises (BvM) theorem serves as the theoretical bridge between these two paradigms, asserting that the posterior distribution asymptotically behaves like a Gaussian centered at an efficient estimator.

While well-understood in parametric models, the BvM theorem faces significant hurdles in nonparametric settings—where the parameter space is infinite-dimensional. The challenge is compounded when the model must be "adaptive," meaning the prior does not have prior knowledge of the underlying function's smoothness. This paper provides a rigorous mathematical framework to establish BvM theorems for such adaptive procedures.

The Problem of Adaptation and "Dishonest" Uncertainty

Nonparametric priors typically require a smoothness parameter $\beta$ to perform optimally. Adaptive priors (such as Empirical Bayes or Hierarchical Bayes) attempt to learn $\beta$ from the data. However, adaptation often leads to "dishonest" uncertainty quantification: the Bayesian might become overly confident in an estimate that is far from the truth.

To overcome this, the author employs the Gaussian White Noise model and introduces a self-similarity condition. This condition ensures that the function's behavior at small scales reflects its behavior at large scales, preventing it from hiding complexity in high-frequency components that the data cannot yet resolve.

Methodology: Weak Topologies and Multiscale Settings

The research utilizes weak topologies to handle the "roughness" of Gaussian white noise in infinite dimensions. Key technical components include:

  • Empirical and Hierarchical Bayes: Methods for estimating hyperparameters directly from data or via hyperpriors.
  • Slab and Spike Priors: Specifically used for $L^\infty$ (supremum norm) settings to manage sparsity and multiscale features.
  • Geometric Considerations: The paper demonstrates that the specific geometry of the credible set (e.g., multiscale balls) is crucial for ensuring frequentist validity.

Key Results and Simulations

The core of the paper is the establishment of BvM theorems (Theorems 3.1, 3.2, and 3.5) across different settings. A significant finding is the asymptotic independence of $L^2$-based and $L^\infty$-based credible sets, suggesting they capture complementary information about the data's frequency components.

Visual representation of an adaptive credible set for a single function realization.

Numerical simulations validate these theoretical findings. As shown in the multi-panel simulations, the proposed adaptive credible sets achieve the desired frequentist coverage as the sample size increases.

Multi-panel simulations demonstrating the consistency and coverage of adaptive multiscale credible sets across different realizations.

These multiscale sets are particularly effective at capturing sharp features (peaks) in the data that traditional smoothing methods might miss.

Conclusion and Future Directions

This work provides a solid theoretical foundation for using Bayesian simulation-based uncertainty in complex nonparametric models. While the self-similarity requirement is a necessary constraint to avoid impossibility results, the framework offers a path forward for more complex models, such as density estimation and non-Gaussian regression. Future research will likely focus on relaxing these conditions and improving the computational efficiency of these high-dimensional simulations.

Summary updated

Abstract

We investigate Bernstein–von Mises theorems for adaptive nonparametric Bayesian procedures in the canonical Gaussian white noise model. We consider both a Hilbert space and multiscale setting with applications in $L^{2}$ and $L^{\infty}$, respectively. This provides a theoretical justification for plug-in procedures, for example the use of certain credible sets for sufficiently smooth linear functionals. We use this general approach to construct optimal frequentist confidence sets based on the posterior distribution. We also provide simulations to numerically illustrate our approach and obtain a visual representation of the geometries involved.
BibTeX
@article{ray2017adaptive,
  author = {Kolyan Ray},
  title = {{Adaptive Bernstein–von Mises theorems in Gaussian white noise}},
  journal = {The Annals of Statistics},
  year = {2017},
  doi = {10.1214/16-aos1533},
}
Figure