Deep Neyman-Scott Processes

Chengkuan Hong, Christian R. Shelton

Abstract

A Neyman-Scott process is a special case of a Cox process where the latent and observable stochastic processes are both Poisson processes. This paper introduces the Deep Neyman-Scott Process (DN-SP), a hierarchical framework where the building components of a network are all Poisson processes. We develop an efficient posterior sampling method via Markov chain Monte Carlo (MCMC) and use it for likelihood-based inference. Our method enables inference in sophisticated hierarchical point processes, demonstrating that additional hidden Poisson layers improve performance for likelihood fitting and event type prediction. Compared to state-of-the-art neural models on temporal real-world datasets, DN-SPs demonstrate competitive data fitting and prediction capabilities while using significantly fewer parameters.

Introduction

Point processes are fundamental mathematical tools for modeling events occurring in time or space, such as seismic activity, high-frequency financial trades, or neural spikes. While the basic Poisson process is often too restrictive due to its independence assumptions, Cox processes offer more flexibility by treating the event rate as a random process itself.

However, existing Cox models face a dilemma: they are either mathematically elegant but overly simplistic (like standard Neyman-Scott processes) or powerful but opaque "black boxes" requiring hundreds of thousands of parameters (like Neural Hawkes Processes). DN-SPs provide a middle ground—a deep, hierarchical model that remains interpretable and highly parameter-efficient.

Methodology

The Hierarchical Framework

The core architecture of a DN-SP involves stacking Temporal Point Processes (TPPs) in layers. The top layer consists of a simple Homogeneous Poisson Process. Each event in a higher layer acts as a "parent" that triggers a cluster of "child" events in the layer below. This triggering mechanism is governed by a Gamma kernel function.

The hierarchical structure of the Deep Neyman-Scott Process, showing the transition from latent layers (z) to the observed layer (x).

Estimating these hidden layers is computationally challenging because, in the posterior distribution, these processes lose their simple temporal ordering and behave like Spatial Point Processes. To address this, the authors utilize Monte Carlo Expectation-Maximization (MCEM) to iteratively update model parameters.

Efficient Inference via Virtual Events

To solve the inference challenge, the authors introduce an auxiliary variable method using "virtual events." These act as candidates for real hidden events, allowing for an efficient MCMC sampler. The sampler employs three primary moves to explore the posterior distribution:

  • Re-sample: Refreshing the pool of virtual candidates.
  • Flip: Converting a virtual event into a real one, or vice versa.
  • Swap: Exchanging a real event with a virtual one to improve the chain's mixing speed.
The MCMC sampling mechanism illustrating Re-sample, Flip, and Swap moves using virtual events (blue) and real events (red).

Experimental Evaluation

The authors evaluated DN-SPs on synthetic data and three diverse real-world datasets: Retweets (2.6M events), Earthquakes (seismic data from 2014–2020), and Homicides (district-level data from Chicago). They compared the model against state-of-the-art neural models, including Neural Hawkes Processes (NHP) and Transformer Hawkes Processes (THP).

Key Insights

  • Depth and Performance: Synthetic tests confirmed that deeper models (more hidden layers) significantly improve log-likelihood and prediction accuracy when the underlying data generation is complex.
  • Extreme Parameter Efficiency: While neural models like THP require hundreds of thousands of parameters to learn event dynamics, DN-SPs achieve comparable or superior results with just tens of parameters.
  • Flexibility: Unlike Hawkes processes, which are strictly causal and temporal, DN-SPs do not enforce a temporal direction. This allows the framework to be naturally extended to purely spatial data.
  • Robustness on Sparse Data: DN-SPs performed particularly well on the Homicides dataset. Neural models often struggle to generalize when data is sparse, whereas the structured hierarchy of the DN-SP provides a helpful inductive bias.

Results and Discussion

On the Earthquakes and Homicides datasets, the DN-SP achieved the lowest Root Mean Square Error (RMSE) for time prediction and the highest accuracy for event type prediction. In terms of log-likelihood, it outperformed all neural baselines on the Retweets and Homicides data.

Despite these strengths, the model has limitations. The current use of a simple Gamma kernel may struggle with non-monotonic triggering patterns. Additionally, while parameter-efficient, the MCMC sampling can be computationally intensive; a 2-hidden-layer model on the Retweets dataset required over 100 hours to train. Future work will focus on designing more complex kernels, accounting for "edge effects" from unobserved parent events, and expanding the model into high-dimensional spatio-temporal domains.

Summary updated

Abstract

A Neyman-Scott process is a special case of a Cox process. The latent and observable stochastic processes are both Poisson processes. We consider a deep Neyman-Scott process in this paper, for which the building components of a network are all Poisson processes. We develop an efficient posterior sampling via Markov chain Monte Carlo and use it for likelihood-based inference. Our method opens up room for the inference in sophisticated hierarchical point processes. We show in the experiments that more hidden Poisson processes brings better performance for likelihood fitting and events types prediction. We also compare our method with state-of-the-art models for temporal real-world datasets and demonstrate competitive abilities for both data fitting and prediction, using far fewer parameters.
BibTeX
@inproceedings{hong2022deep,
  author = {Chengkuan Hong and Christian R. Shelton},
  title = {{Deep Neyman-Scott Processes}},
  booktitle = {International Conference on Artificial Intelligence and Statistics},
  year = {2022},
  url = {https://openreview.net/forum?id=0DtmoBcuXX},
}
Figure