Rank-Based Mixture Models for Temporal Point Processes

Yang Chen, Yijia Ma, Wei WuFrontiers in Applied Mathematics and Statistics 2022 · DOI

TL;DR

Researchers have developed a new way to model temporal point processes—sequences of events over time—by focusing on the "big picture" rather than just the next event. By modeling the distribution of inter-event times using Dirichlet and Gaussian mixture models, they can identify "typical" event patterns and outliers, significantly improving classification performance on complex, multi-modal data like neural spike trains.

What Problem Does This Paper Address?

Temporal Point Processes (TPPs) are mathematical frameworks used to describe events occurring at random times, such as earthquakes, stock market trades, or neurons firing in the brain. Traditionally, these are modeled using a "conditional intensity function," which predicts the probability of an event happening now based on what happened in the past.

While this "evolutionary" approach is great for prediction, it struggles to capture the overall structure of a sequence. It’s hard to look at a whole timeline and ask: "Is this a typical sequence for this process, or is it an outlier?" or "How does this sequence rank compared to others?" This paper addresses the gap in "center-outward ranking"—the ability to define a central, representative pattern for a set of timelines and measure how far others deviate from it.

Methods

The authors propose a data-driven framework that decomposes the likelihood of a point process into two factors: the number of events and the conditional distribution of those events. The core innovation lies in how they model the Inter-Event Times (IETs).

Because IETs are non-negative and sum to a constant total time ($T$), they naturally exist on a mathematical structure called a simplex. The authors propose two ways to handle this:

  1. Dirichlet Mixture Model (DMM): They model the IETs directly using a mixture of Dirichlet distributions. Since Dirichlet distributions live on the simplex, they are a natural fit, but they lack "closed-form" updates, making them computationally heavier to estimate.
  2. Gaussian Mixture Model (GMM) with Transformation: To simplify things, they use an Isometric Logratio (ilr) transformation. This bijectively maps the constrained simplex data into unconstrained Euclidean space. Once transformed, they can use standard, highly efficient Gaussian Mixture Models.

To make these models truly data-driven, they use a Dirichlet Process (DP) to automatically determine the optimal number of mixture components (clusters) in the data, followed by an Expectation-Maximization (EM) algorithm to fine-tune the parameters.

Datasets

The authors validated their approach using three distinct sources:

  • Homogeneous Poisson Process (HPP): A baseline simulation where events occur at a constant average rate.
  • Inhomogeneous Poisson Processes (IPPs): Simulations with four different time-varying intensity functions, designed to create complex, multi-modal event patterns.
  • Real-World Neural Data: Spike trains recorded from the primary motor cortex of a macaque monkey performing a "square-path" reaching task. This dataset consists of 240 trials across four different movement paths.

Key Insights and Ideas

  • From Local to Global: By shifting focus from the intensity function to the joint distribution of IETs, the model captures "global" features of the process that traditional models miss.
  • The Power of Mixtures: Single distributions (like a single Dirichlet) fail when the data has multiple "typical" patterns. Mixture models allow the framework to represent processes that might have several different ways of being "normal."
  • Simplex to Euclidean Mapping: The use of the ilr transformation is a clever "aha" moment. It allows researchers to use well-understood Gaussian tools on data that is physically constrained (times summing to a total), bypassing the mathematical headaches of the Dirichlet model's lack of conjugate priors.
  • Center-Outward Ranking: The model provides a "pseudo-likelihood" that acts as a depth function. Realizations with high likelihood are "central" (typical), while those with low likelihood are outliers.

Key Results

  • Superior Classification: In simulated multi-modal processes (IPPs), the mixture models achieved significantly higher classification accuracy than single-component models, which simply couldn't "see" the multiple clusters in the data.
  • Neural Decoding: On the real-world macaque dataset, the models were tasked with identifying which of the four paths the monkey was drawing based solely on neural spikes.
  • The Dirichlet Mixture Model achieved 88.33% accuracy.
  • The Gaussian Mixture Model (via ilr transformation) achieved 86.67% accuracy.
  • Robustness: Both mixture models performed comparably to state-of-the-art nearest neighbor methods, proving that this probabilistic approach is both accurate and provides a more interpretable "rank" of the data.

Limitations and Future Work

The authors acknowledge a few hurdles:

  • Computational Complexity: The Dirichlet model, while "purer" as it stays on the simplex, is more expensive to compute because it lacks closed-form solutions for certain steps.
  • Sensitivity to Outliers: While the GMM is faster, the ilr transformation can sometimes map points near the boundaries of the simplex to very sparse regions in Euclidean space, making the model sensitive to those specific outliers.
  • Future Directions: The authors suggest exploring other modeling approaches on the transformed IETs beyond Gaussian mixtures and investigating how this framework could be specifically tuned for automated outlier detection in large-scale temporal datasets.

Summary updated

Abstract

Temporal point process, an important area in stochastic process, has been extensively studied in both theory and applications. The classical theory on point process focuses on time-based framework, where a conditional intensity function at each given time can fully describe the process. However, such a framework cannot directly capture important overall features/patterns in the process, for example, characterizing a center-outward rank or identifying outliers in a given sample. In this article, we propose a new, data-driven model for regular point process. Our study provides a probabilistic model using two factors: (1) the number of events in the process, and (2) the conditional distribution of these events given the number. The second factor is the key challenge. Based on the equivalent inter-event representation, we propose two frameworks on the inter-event times (IETs) to capture large variability in a given process—One is to model the IETs directly by a Dirichlet mixture, and the other is to model the isometric logratio transformed IETs by a classical Gaussian mixture. Both mixture models can be properly estimated using a Dirichlet process (for the number of components) and Expectation-Maximization algorithm (for parameters in the models). In particular, we thoroughly examine the new models on the commonly used Poisson processes. We finally demonstrate the effectiveness of the new framework using two simulations and one real experimental dataset.
BibTeX
@inproceedings{chen2022rank,
  author = {Yang Chen and Yijia Ma and Wei Wu},
  title = {{Rank-Based Mixture Models for Temporal Point Processes}},
  booktitle = {Frontiers in Applied Mathematics and Statistics},
  year = {2022},
  doi = {10.3389/fams.2022.852314},
}
Figure