Nonparametric Bayesian analysis of the compound Poisson prior for support boundary recovery

Markus Reiß, Johannes Schmidt-Hieber

Abstract

Given data from a Poisson point process with intensity $(x,y) \mapsto n \mathbf{1}(f(x) \leq y)$, frequentist properties for the Bayesian reconstruction of the support boundary function $f$ are derived. We mainly study compound Poisson process priors with fixed intensity proving that the posterior contracts with nearly optimal rate for monotone support boundaries and adapts to Hölder smooth boundaries. We then derive a limiting shape result for a compound Poisson process prior and a function space with increasing parameter dimension. It is shown that the marginal posterior of the mean functional performs an automatic bias correction and contracts with a faster rate than the MLE. In this case, $(1-\alpha)$-credible sets are also asymptotic $(1-\alpha)$-confidence intervals. As a negative result, it is shown that the frequentist coverage of credible sets is lost for linear functions $f$ outside the function class.

Introduction

In fields ranging from image processing to econometrics, researchers often face the challenge of recovering the "edge" or support boundary of a dataset. Consider a scenario where data points appear only above an unknown curve $f(x)$. This is a non-regular statistical problem because the "errors" are one-sided: data points are strictly on one side of the boundary. Consequently, standard estimators like the Maximum Likelihood Estimator (MLE) exhibit unusual behavior, such as "overshooting" the true boundary and failing to follow a Gaussian distribution. This paper investigates whether a Nonparametric Bayesian approach, specifically using Compound Poisson Process (CPP) priors, can provide more robust recovery and uncertainty quantification than classical methods.

Methodology

The authors model the data as a Poisson Point Process (PPP) with an intensity that is zero below the boundary $f(x)$ and $n$ above it. To estimate $f$, they employ Compound Poisson Process (CPP) priors, which treat the unknown boundary as a piecewise constant function. In this setup, the number of "jumps" follows a Poisson distribution, jump locations are uniform, and jump sizes are i.i.d.

Illustration of the piecewise constant Compound Poisson Process (CPP) prior and the resulting credible regions for a monotone boundary.

This approach is computationally efficient and allows the researchers to analyze the posterior contraction rate—the speed at which the Bayesian estimate converges to the true boundary as the sample size $n$ increases. The study also explores a generalized Bernstein–von Mises theorem to relate Bayesian credible intervals to frequentist confidence intervals.

Key Insights and Results

The analysis reveals several critical advantages of the Bayesian approach:

  1. Adaptivity: The CPP prior automatically adapts to the unknown smoothness of the boundary. It achieves nearly optimal recovery rates—approximately $(\log n / n)^{\beta/(1+\beta)}$—for both $\beta$-Hölder smooth and monotone boundaries without requiring manual hyperparameter tuning.
  2. Automatic Bias Correction: Classical MLEs for boundaries are inherently biased because they are constrained by the lowest observed data points. The authors demonstrate that the Bayesian marginal posterior for the mean functional naturally shifts downwards, performing an automatic bias correction.
  3. Outperforming the MLE: For piecewise constant monotone functions, the Bayesian approach contracts at a faster rate than the uncorrected MLE. Furthermore, $(1-\alpha)$-credible sets are shown to be valid asymptotic $(1-\alpha)$-confidence intervals.
Comparison of the MLE (red) and posterior draws (grey) for a smooth boundary (black curve). The MLE typically overshoots the boundary, while Bayesian draws provide a more nuanced estimation.

However, the paper also identifies a "Linear Trap": if the true boundary is a smooth linear function but a piecewise constant prior is used, the frequentist coverage of Bayesian credible intervals is lost. This highlights the sensitivity of Bayesian methods to model misspecification in boundary problems.

Conclusion

This work provides a rigorous frequentist foundation for using CPP priors in support boundary recovery. While the Bayesian approach offers significant benefits like automatic bias correction and adaptivity, care must be taken regarding the choice of prior when the true boundary is expected to be smoother than the prior's sample paths. Future research may extend these guarantees to higher-dimensional surfaces and explore smoother prior constructions to mitigate misspecification issues.

Summary updated

Abstract

Given data from a Poisson point process with intensity $(x,y)\mapston\mathbf{1}(f(x)\leq y)$, frequentist properties for the Bayesian reconstruction of the support boundary function $f$ are derived. We mainly study compound Poisson process priors with fixed intensity proving that the posterior contracts with nearly optimal rate for monotone support boundaries and adapts to Hölder smooth boundaries. We then derive a limiting shape result for a compound Poisson process prior and a function space with increasing parameter dimension. It is shown that the marginal posterior of the mean functional performs an automatic bias correction and contracts with a faster rate than the MLE. In this case, $(1-\alpha )$-credible sets are also asymptotic $(1-\alpha )$-confidence intervals. As a negative result, it is shown that the frequentist coverage of credible sets is lost for linear functions $f$ outside the function class.
BibTeX
@article{rei2020nonparametric,
  author = {Markus Reiß and Johannes Schmidt-Hieber},
  title = {{Nonparametric Bayesian analysis of the compound Poisson prior for support boundary recovery}},
  journal = {The Annals of Statistics},
  year = {2020},
  doi = {10.1214/19-aos1853},
}
Figure