Neural Temporal Point Processes

80 concepts60 papers

Real-world data is frequently asynchronous, continuous, and irregularly timed. Consider the arrival patterns of earthquakes, high-frequency financial trades, user clicks on a website, or patient visits to a hospital. Traditional deep learning architectures handle sequential data by treating it as a series of discrete, uniform steps ($t_1, t_2, t_3, \dots$), effectively discarding the precise gaps between events. Alternatively, practitioners resort to manual temporal binning, which destroys fine-grained temporal information and struggles when events are highly sparse or bursty.

To model these event streams natively without information loss, statisticians rely on Temporal Point Processes (TPPs). As surveyed by Angelica Liguori et al. in Modeling Events and Interactions through Temporal Processes - A Survey, the modern intersection of deep learning and point processes has yielded Neural Temporal Point Processes. These models replace rigid, hand-engineered statistical assumptions with flexible neural networks, allowing systems to learn the complex, non-linear dynamics of real-world event streams directly from data.

The Intuition of Intensity

The mathematical cornerstone of any temporal point process is its conditional intensity function, denoted as $\lambda(t | \mathcal{H}_t)$. Intuitively, the intensity represents the instantaneous arrival rate of an event at time $t$, conditioned on the history of all past events $\mathcal{H}_t$ up to that moment.

$$\lambda(t | \mathcal{H}_t) = \lim_{\Delta t \to 0} \frac{\mathbb{P}(\text{Event in } [t, t + \Delta t] | \mathcal{H}_t)}{\Delta t}$$

In classical statistical models, such as the self-exciting Hawkes process, this intensity is governed by a fixed parametric formula. For example, a past event triggers an immediate spike in intensity that decays exponentially over time:

$$\lambda(t) = \mu + \sum_{t_i < t} \alpha e^{-\beta (t - t_i)}$$

While mathematically elegant, this formulation forces a rigid assumption onto the data: every event must be self-exciting, and its influence must decay monotonically. In reality, event dynamics are rarely this simple. A tweet might trigger an immediate wave of retweets, but a medical intervention might inhibit a future symptom, or a scheduled calendar event might exhibit a delayed, non-monotonic influence that peaks hours later.

To capture these intricate patterns, Shuai Xiao et al. proposed parameterizing the intensity function using Recurrent Neural Networks (RNNs) in Modeling the Intensity Function of Point Process Via Recurrent Neural Networks. Instead of prescribing a fixed mathematical decay kernel, an RNN processes the historical sequence of events to maintain a hidden state $h_i$ immediately following the $i$-th event. The continuous-time intensity function for any time $t > t_i$ is then computed as a non-linear projection of both the historical summary $h_i$ and the continuously changing time gap $(t - t_i)$. This allows the model to learn arbitrary excitation or inhibition patterns directly from data.

Event History:  (t_1) ------> (t_2) ------> (t_3) 
                  |             |             |
RNN Hidden State: h_1 --------> h_2 --------> h_3
                                              |
                                              v
Continuous Time t > t_3:             \lambda(t) = f(h_3, t - t_3)

Moving Beyond Recurrence: Attention and Spectral Kernels

While recurrent architectures successfully move away from fixed exponential decay, they struggle with long-range dependencies and non-monotonic influences. If an event's primary impact occurs after a specific, learned delay, an RNN must compress this temporal relationship entirely into its fixed-size hidden state vector.

To overcome this limitation, Simiao Zuo et al. introduced the Transformer Hawkes Process (THP) in Linking Transformer to Hawkes Process for Information Cascade Prediction. By replacing recurrence with self-attention, the model can look back across the entire history of events simultaneously. The discrete hidden representations generated by the Transformer are mapped to a continuous-time intensity function using a softplus-based transformation, enabling the model to evaluate the likelihood of an event at any arbitrary point in time. Similarly, Dongxia Wu et al. leveraged instance-wise self-attention in Learning Granger Causality from Instance-wise Self-attentive Hawkes Processes to capture non-linear decay patterns, making it possible to explicitly unearth Granger causal relationships between individual events.

An alternative approach to capturing complex temporal dependencies is to operate in the frequency domain. In Deep Fourier Kernel for Self-Attentive Point Processes, Shixiang Zhu et al. developed a framework (DAPP) that replaces standard dot-product attention scores with a shift-invariant kernel. Instead of selecting a standard kernel function, they train a deep neural network to learn the spectral density of the kernel. By learning the kernel's spectrum, the model can approximate highly complex, multi-modal, and non-monotonic event influences, capturing delayed reactions that traditional models miss entirely.

The Integration Bottleneck and the Cumulative Hazard Trick

If neural networks are so adept at modeling arbitrary functions, why can we not simply parameterize $\lambda(t)$ with a standard, unconstrained multi-layer perceptron? The answer lies in the mathematical formulation of the training objective.

To train a temporal point process via maximum likelihood estimation, we must maximize the joint probability density of observing a specific sequence of events over a total time horizon $[0, T]$. The log-likelihood function is defined as:

$$\log L = \sum_{i=1}^N \log \lambda(t_i | \mathcal{H}_{t_i}) - \int_0^T \lambda(t | \mathcal{H}_t) dt$$

The left-hand term maximizes the intensity at the exact moments the events actually occurred. The right-hand term—the continuous integral of the intensity function over the entire time horizon—is known as the cumulative hazard. This integral represents the survival probability, accounting for the fact that no events occurred during the intervals between the observed timestamps.

If $\lambda(t)$ is parameterized by an arbitrary, highly non-linear neural network, this continuous integral becomes mathematically intractable. It possesses no closed-form analytical solution. Early neural TPP models were forced to rely on numerical integration techniques, such as Monte Carlo integration or Riemann sums. These methods are computationally expensive, scale poorly to long sequences, and introduce approximation errors that destabilize training.

To bypass this bottleneck entirely, Takahiro Omi and Naonori Ueda introduced an elegant inversion in Fully Neural Network based Model for General Temporal Point Processes. Instead of using a neural network to model the instantaneous intensity $\lambda(t)$ and struggling to integrate it, they designed a neural network to directly model the integral itself—the cumulative hazard function $\Lambda(t) = \int_0^t \lambda(\tau) d\tau$.

Traditional Approach:  Neural Network ---> \lambda(t) ---> [Intractable Numerical Integration] ---> \Lambda(t)
Omi & Ueda Approach:   Neural Network ---------------------------------------------------------> \Lambda(t)
                                                                                                    |
                                                                                     [Automatic Differentiation]
                                                                                                    v
                                                                                                \lambda(t)

By utilizing a strictly positive and monotonically increasing neural network architecture to output $\Lambda(t)$, the cumulative hazard term in the log-likelihood function is obtained instantly without any integration. To find the instantaneous intensity $\lambda(t)$ needed for the first term of the log-likelihood, the authors simply take the derivative of the network's output with respect to time:

$$\lambda(t) = \frac{d}{dt}\Lambda(t)$$

Because modern deep learning frameworks are built on automatic differentiation, calculating this derivative is exact, instantaneous, and computationally efficient. This architectural trick eliminates the integration bottleneck entirely, allowing neural TPPs to scale efficiently to massive datasets.

An alternative solution to this bottleneck involves using a non-parametric mixture of kernels. In Neural Point Process for Learning Spatiotemporal Event Dynamics, Zihao Zhou et al. structured the intensity function as a mixture of base kernels whose parameters (such as magnitude, spatial bandwidth, and temporal decay) are predicted by a neural network. Because the underlying base kernels have known analytical integrals, the model retains the flexibility of deep learning while preserving a closed-form solution for the cumulative hazard.

Extending the Intensity Paradigm

The flexibility of neural intensity functions has allowed researchers to extend continuous-time event modeling into higher dimensions:

  • Spatiotemporal Point Processes: In real-world scenarios, events do not just happen at a specific time; they happen at a specific place. As detailed by Sumantrak Mukherjee et al. in Neural Spatiotemporal Point Processes: Trends and Challenges, neural networks can parameterize the spatial shape and scale of triggering kernels. This allows models to adapt to complex geographic boundaries or urban traffic networks, as demonstrated by Maya Okawa et al. in Deep Mixture Point Processes.
  • Relational and Graph Streams: Multi-dimensional event streams often occur across interconnected networks, such as dynamic knowledge graphs or social networks. Yunhao Zhang and Junchi Yan demonstrated in Neural Relation Inference for Multi-dimensional Temporal Point Processes via Message Passing Graph that integrating message-passing graph neural networks directly into the intensity function allows models to capture complex, non-linear relational dependencies across asynchronous event sequences.
  • Survival Analysis: The neural intensity framework also provides a powerful alternative to traditional survival analysis. By utilizing a recurrent neural network to parameterize the conditional hazard function, Kan Ren et al. showed in Deep Recurrent Survival Analysis that models can remain completely distribution-agnostic, accommodating highly irregular and complex temporal survival patterns without relying on rigid parametric assumptions like Weibull or Exponential distributions.

Key Foundational Work

A Semiparametric Bayesian Model for Detecting Synchrony Among Multiple Neurons (2014) — contributed 1 concept elements to this direction.
A Model for Temporal Dependencies in Event Streams (2011) — contributed 1 concept elements to this direction.
History & citationVersion 2 · 2026-10-04

Current · Permanent link

  1. Version 2 (latest) · 2026-10-04 · 60 papers
  2. Version 1 · 2026-10-04 · 72 papers
Cite this direction

Latest version (a living page):

@misc{lacuna_neural_intensity_function_809ad613,
  title = {neural intensity function},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/neural-intensity-function/rdr_809ad6133b9b4d3291343b5d1a8d75fe},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 2, frozen text):

@misc{lacuna_neural_intensity_function_809ad613_v2,
  title = {neural intensity function},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/neural-intensity-function/rdr_809ad6133b9b4d3291343b5d1a8d75fe/v/2},
  note = {Version 2, frozen 2026-10-04. Accessed 2026-11-20},
}

See something wrong? Report a mistake