Many real-world phenomena arrive as sequences of discrete events occurring at irregular intervals: earthquakes and their aftershocks, high-frequency financial trades, user clicks on a website, or viral retweets on social media. To model these asynchronous event streams, statisticians traditionally rely on Temporal Point Processes (TPPs).
A TPP is fully characterized by its conditional intensity function, $\lambda(t | \mathcal{H}_t)$, which represents the instantaneous rate of an event occurring at time $t$ given the historical sequence of past events $\mathcal{H}_t$. Mathematically, it is defined as:
$$\lambda(t | \mathcal{H}_t) = \lim_{dt \to 0} \frac{\mathbb{P}(\text{Event in } [t, t + dt] | \mathcal{H}_t)}{dt}$$
Historically, the field relied heavily on fixed parametric forms for $\lambda(t)$. For instance, the classic self-exciting Hawkes process assumes that past events increase the likelihood of future events via an explicitly defined kernel, typically exhibiting exponential decay:
$$\lambda(t) = \mu + \sum_{t_i < t} \alpha e^{-\beta (t - t_i)}$$
While mathematically elegant, this formulation suffers from parametric rigidity. Real-world dynamics rarely conform to simple exponential curves. Influence might experience a delayed peak, exhibit periodic oscillations, or even act inhibitorily (where a past event suppresses future occurrences). Forcing data into rigid parametric boxes leads to severe model misspecification, as noted by Learning Conditional Generative Models for Temporal Point Processes.
To overcome this, a powerful paradigm shift has emerged: replacing hand-crafted parametric kernels with nonparametric estimators and deep neural networks that allow the data to dictate the true shape of temporal influence.
Breaking the Parametric Box with Nonparametric Kernels
The first natural step away from rigid assumptions is to treat the triggering kernel not as a fixed mathematical formula, but as an arbitrary function $g(t)$ to be learned directly from data.
Parametric (Hawkes): Intensity spikes, then forces an exponential decay curve.
Nonparametric Kernel: Intensity can adapt to delayed peaks, plateaus, or complex rhythms.
Ke Zhou et al. pioneered this direction in Learning Triggering Kernels for Multi-dimensional Hawkes Processes by formulating an infinite-dimensional optimization problem. By combining Majorization-Minimization with Euler-Lagrange equations, the authors allowed the data to freely shape the triggering kernels, successfully capturing complex temporal patterns like delayed peaks and periodic trends that exponential models miss entirely.
Another elegant way to achieve this flexibility is by projecting the triggering functions into a Reproducing Kernel Hilbert Space (RKHS). In Online Learning for Multivariate Hawkes Processes, Yingxiang Yang demonstrated that an RKHS framework enables nonparametric online learning. This approach allows the model to adapt in real-time to heavy-tailed distributions or complex event-triggering patterns without suffering from model mismatch errors.
Neurally Self-Modulating Intensity
While nonparametric kernels offer flexibility, they still typically assume a linear superposition of past influences: the total intensity is simply the sum of individual historical effects. This assumption breaks down when events interact non-linearly or when the context of an event changes its downstream impact.
Neural TPPs solve this by using deep neural networks to encode the event history $\mathcal{H}_t$ into a continuous latent state $h(t)$, which then dynamically modulates the intensity function.
History (t_1, t_2, ...) ──> RNN / Transformer ──> Latent State h(t) ──> Non-linear f(h(t)) ──> Intensity λ(t)
Shuai Xiao et al. (Modeling the Intensity Function of Point Process Via Recurrent Neural Networks) and Hongyuan Mei et al. (The Neural Hawkes Process: A Neurally Self-Modulating Multivariate Point Process) introduced architectures where a Recurrent Neural Network (RNN) maintains a hidden state that updates discretely when an event occurs, but decays continuously between events. The intensity is then computed as a non-linear projection of this continuously evolving hidden state:
$$\lambda(t) = f(\mathbf{w}^T \mathbf{h}(t))$$
This allows the model to capture intricate history-dependent dynamics, including inhibitory effects and complex non-monotonic temporal profiles. To scale these models and capture long-range dependencies more effectively, Simiao Zuo et al. proposed the Transformer Hawkes Process, replacing the recurrent bottleneck with self-attention mechanisms across asynchronous event sequences.
The Integration Bottleneck and How to Solve It
While neural networks provide unparalleled flexibility in representing $\lambda(t)$, they introduce a severe mathematical challenge during training. The log-likelihood of an observed event sequence over an interval $[0, T]$ is given by:
$$\log \mathcal{L} = \sum_{i=1}^N \log \lambda(t_i) - \int_{0}^T \lambda(t) dt$$
The first term evaluates the intensity at the exact moments events occurred, which is straightforward. However, the second term—the survival term—requires integrating the intensity function over the entire continuous interval. Because a neural network output is free-form, this integral does not have a closed-form analytical solution.
Early neural TPPs relied on numerical integration techniques (like Monte Carlo sampling or Riemann sums), which are computationally expensive, slow down training, and introduce approximation errors. Modern architectures bypass this bottleneck using two main strategies:
1. Modeling the Cumulative Hazard Directly
Instead of parameterizing the intensity function $\lambda(t)$ and struggling to integrate it, why not parameterize the integral itself?
Takahiro Omi proposed this elegant workaround in Fully Neural Network based Model for General Temporal Point Processes. The model uses a neural network to directly output the cumulative hazard function $\Lambda(t) = \int_{0}^t \lambda(\tau) d\tau$. To ensure mathematical validity, the network is constrained to be strictly increasing and positive. The actual intensity function is then obtained effortlessly via automatic differentiation:
$$\lambda(t) = \frac{d\Lambda(t)}{dt}$$
This inversion makes the log-likelihood calculation exact, fast, and completely free of numerical integration artifacts.
2. Analytical Mixture Kernels
An alternative strategy is to restrict the output layer of the neural network to a family of functions that can be integrated analytically. For instance, in Neural Point Process for Learning Spatiotemporal Event Dynamics, Zihao Zhou et al. combined a Transformer-based history encoder with a non-parametric mixture of kernels. Because the underlying components of the mixture have known antiderivatives, the model achieves exact closed-form integration while training up to 100x faster than numerical alternatives.
Continuous-Time Latent Dynamics via ODEs and SDEs
When event sequences are driven by underlying physical systems or continuous environmental drift (such as changing weather patterns affecting wildfire occurrences), modeling the latent state as a simple decaying vector is insufficient.
To capture these continuous-time trajectories, researchers have fused TPPs with Neural Ordinary Differential Equations (ODEs) and Stochastic Differential Equations (SDEs).
Discrete Event ──> Jump in Latent State ──> Continuous Evolution via Neural SDE ──> Next Event
Ricky T. Q. Chen et al. leveraged this in Neural Spatio-Temporal Point Processes, utilizing continuous-time normalizing flows to track complex, multimodal spatial and temporal distributions simultaneously.
To account for stochastic volatility and unpredictable environmental shocks between events, Junteng Jia et al. developed Neural Jump Stochastic Differential Equations. This framework embeds discrete, event-driven jumps directly into a continuous latent flow governed by an SDE. By parameterizing the drift, diffusion, and jump intensities with neural networks, the model learns free-form, data-driven dynamics that accurately mirror the messy, non-linear realities of complex systems.