Most machine learning models are designed for "grid-like" data: images have fixed pixel layouts, and time series are usually sampled at regular intervals (e.g., daily stock prices). However, real-world events—like a patient’s hospital visits, a user’s social media posts, or a series of high-frequency trades—happen at irregular, asynchronous intervals.
To model these "event streams," researchers have turned to Temporal Point Processes (TPPs). Traditionally, these models relied on rigid mathematical formulas to describe how past events influence the future. A new research direction replaces these fixed formulas with neural networks, creating Neural Temporal Point Processes (NTPPs) that can learn the complex, non-linear rhythms of human and natural activity directly from data.
The Intensity Function as a Neural Network
The core of any point process is the conditional intensity function, denoted as $\lambda(t)$. This function represents the instantaneous probability that an event will occur at time $t$, given everything that has happened before.
In classical models like the Hawkes process, $\lambda(t)$ is defined by a fixed "kernel"—usually an exponential decay. This assumes that the influence of a past event on the future always fades in a specific, predictable way. As Shuai Xiao and Hongyuan Zha argue in Modeling the Intensity Function of Point Process Via Recurrent Neural Networks, these parametric assumptions often fail to capture the "messy" dynamics of real-world data.
The breakthrough in this field is the use of neural networks to parameterize $\lambda(t)$. Instead of prescribing a decay shape, researchers use Recurrent Neural Networks (RNNs) or Transformers to encode the history of events into a hidden state $h$. This state is then mapped to a continuous-time function that can represent arbitrary patterns—including inhibition (where one event makes another less likely) or complex periodicities.
From Sequences to Continuous Latent Flows
While early models like CYAN-RNN, developed by Huawei Shen and colleagues in Cascade Dynamics Modeling with Attention-based Recurrent Neural Network, focused on capturing cross-dependencies in sequences, the field has moved toward treating time as truly continuous.
Rather than just predicting the "next step," models now evolve a latent state through time using Neural Ordinary Differential Equations (ODEs). In Neural Jump Stochastic Differential Equations, Junteng Jia and Austin R. Benson demonstrate how to model systems that evolve smoothly but are interrupted by stochastic "shocks." This allows the model to maintain a "memory" that changes even during the gaps between events, which is crucial for domains like healthcare where the relevance of a medical record decays over time (MedGraph).
Universal Approximation of Event Dynamics
A major theoretical milestone in this direction is the proof that neural architectures can represent any valid point process. In UNIPoint: Universally Approximating Point Processes Intensities, Lexing Xie and Alexander Soen introduced a framework that uses a sum of basis functions weighted by an RNN.
This architecture provides a mathematical guarantee: if a pattern exists in the timing of events, a sufficiently large neural point process can learn it. This moves the field away from "feature engineering" (trying to find the right mathematical decay) toward "architectural engineering" (designing networks that can efficiently learn those decays).
Capturing Structure and Space
The flexibility of neural intensity functions has allowed researchers to extend point processes into higher dimensions:
- Spatio-Temporal Modeling: Maya Okawa in Deep Mixture Point Processes uses neural networks to learn how events like taxi pickups or crimes are distributed across both time and city geography.
- Relational Graphs: The Graph Hawkes Transformer, proposed by Haohai Sun and Han Hu, applies self-attention to temporal knowledge graphs, predicting not just when an event will happen, but which entities in a network will be involved (Graph Hawkes Transformer for Extrapolated Reasoning on Temporal Knowledge Graphs).
- Causal Discovery: Models like TNPAR by Ruichu Cai and Wei Chen use neural point processes to infer Granger causality, helping to distinguish between events that simply happen together and events that actually trigger one another (TNPAR: Topological Neural Poisson Auto-Regressive Model for Learning Granger Causal Structure from Event Sequences).
By replacing rigid kernels with learned neural functions, this research direction has transformed point processes from a niche statistical tool into a powerful, general-purpose framework for understanding the asynchronous pulse of the real world.