Optimality of Poisson processes intensity learning with Gaussian processes

Alisa Kirichenko, Harry van ZantenJMLR 2015

TL;DR

This paper provides theoretical guarantees for a "Sigmoidal Gaussian Cox Process" (SGCP) method for learning the intensity of Poisson processes. It proves that this method can automatically adapt to the smoothness of the underlying intensity function and achieve optimal convergence rates, a significant theoretical underpinning for a previously proposed practical approach.

What Problem Does This Paper Address?

In many scientific and engineering fields, we encounter data that can be modeled as point patterns, such as the locations of stars in the sky, the spread of a disease, or the distribution of defects on a surface. Inhomogeneous Poisson processes are a common tool for modeling such data, where the rate at which points occur varies across space. A key challenge is to accurately estimate this underlying "intensity function" from observed point data.

Traditional methods often rely on strong assumptions about the intensity function's form or require manual tuning of smoothing parameters. This can lead to either overfitting (capturing noise) or underfitting (oversmoothing and missing important details). The ideal solution would be a method that can automatically adjust its "smoothness" based on the data and the true underlying intensity, achieving the best possible accuracy for any given level of smoothness. This paper tackles the theoretical justification for such an adaptive method.

Methods

The paper focuses on the "Sigmoidal Gaussian Cox Process" (SGCP) approach, which models the intensity function as a transformed Gaussian Process (GP). Specifically, the intensity $\lambda(s)$ is expressed as $\lambda(s) = \lambda^* \sigma(g(s))$, where $\lambda^*$ is a multiplicative constant, $\sigma$ is a sigmoid (logistic) function, and $g(s)$ is a Gaussian Process. The SGCP method is hierarchical: it places priors not only on the GP itself but also on its hyperparameters (like the length scale) and the multiplicative constant $\lambda^*$.

The core of the paper's contribution lies in its theoretical analysis of the posterior contraction rate of this SGCP model. This means they analyze how quickly the estimated intensity function converges to the true underlying intensity function as more data becomes available. They employ a sophisticated Bayesian nonparametrics framework, building upon existing results for Gaussian Processes. The key technical challenge is to show that the SGCP model, with carefully chosen priors on its hyperparameters, can achieve optimal convergence rates for any level of smoothness of the true intensity function, without needing to know that smoothness level beforehand. This involves proving conditions related to the "prior mass" near the true function and the "entropy" of the prior over certain regions of the function space.

Datasets

The paper does not introduce or use specific datasets for training or evaluation. Instead, it focuses on theoretical analysis and relies on simulated data implicitly through the mathematical framework of Poisson processes.

Key Insights and Ideas

  • Automatic Adaptation is Key: The most significant insight is the theoretical demonstration that the SGCP model, with appropriate prior choices, can automatically adapt to the unknown smoothness of the intensity function. This means it doesn't require manual tuning of smoothing parameters and can achieve the best possible learning rate for any given smoothness level.
  • Theoretical Foundation for a Practical Method: The paper provides crucial theoretical backing for the SGCP approach, which was previously proposed and shown to work well in simulations and real-world examples. This theoretical guarantee enhances confidence in the method's robustness and performance.
  • The Power of Hierarchical Priors: The study highlights how carefully chosen hierarchical priors on the hyperparameters of the Gaussian Process and the multiplicative constant are essential for achieving this adaptive behavior and optimal convergence rates.
  • Bridging Theory and Practice: The work bridges a gap between practical, computationally efficient methods (like the original SGCP) and rigorous theoretical guarantees, making the method more appealing for researchers and practitioners.

Key Results

The main result is Theorem 3.1, which states that under certain conditions on the priors, the SGCP posterior distribution contracts around the true intensity function $\lambda_0$ at an optimal rate of $n^{-\beta/(d+2\beta)}$ (up to a logarithmic factor), where $n$ is the number of data points, $d$ is the dimensionality of the domain, and $\beta$ is the smoothness of $\lambda_0$. This rate is rate-adaptive, meaning it automatically adjusts to the unknown smoothness $\beta$. The paper also establishes conditions on the priors (Sections 4.1 and 4.4) and the construction of "sieves" (approximating sets) (Section 4.2) that guarantee this optimal contraction.

Limitations and Future Work

  • Upper Bounds: The results provide upper bounds on the contraction rate. While this is standard in theoretical analysis, it doesn't definitively prove that these rates are tight (i.e., the absolute best possible). The authors suggest that further research, potentially building on work by Castillo (2008) and van der Vaart and van Zanten (2011), could explore this.
  • Generalizations: The authors explicitly mention several avenues for future work:
  • Anisotropic Smoothness: Extending the results to anisotropic smoothness classes and priors, as considered in Bhattacharya et al. (2014).
  • Analytic Functions: Applying the framework to classes of analytic functions, as studied in van der Vaart and van Zanten (2009).
  • Flexibility in Priors and Kernels: While the paper shows some flexibility in choosing priors and GP kernels, the authors note that deviating too far from the imposed conditions might lead to sub-optimal performance. Further research could precisely delineate these boundaries.
  • Other Link Functions: The paper uses a smooth, strictly increasing link function, and the results hold for the sigmoid function. Investigating the impact of other link functions would be valuable.

Summary updated

BibTeX
@article{kirichenko2015optimality,
  author = {Alisa Kirichenko and Harry van Zanten},
  title = {{Optimality of Poisson processes intensity learning with Gaussian processes}},
  journal = {JMLR},
  year = {2015},
}
Figure