TL;DR
The supposed performance gap between classical Hawkes processes and modern neural models is largely an artifact of comparing "weak" classical baselines against "strong" neural architectures. By equipping Hawkes processes with flexible spline-based components, they become competitive with—and sometimes outperform—neural models on standard benchmarks, revealing that the primary advantage of neural TPPs lies in modeling when events occur rather than which type of event happens.
What Problem Does This Paper Address?
Temporal Point Processes (TPPs) are the mathematical workhorse for modeling event sequences in continuous time, such as social media interactions, seismic activity, or patient journeys in a hospital. For decades, the Hawkes process was the standard, modeling events as a sum of a baseline rate and "triggering" effects from past events.
Recently, the research community shifted toward Neural TPPs, which use deep learning (RNNs, Transformers) to model complex, non-linear dependencies. Recent benchmarks suggested that Neural TPPs significantly outperform Hawkes processes. However, the authors of this paper noticed a flaw in the comparison: researchers were pitting highly flexible neural models against the most restrictive, "vanilla" versions of Hawkes processes (constant baselines and simple exponential kernels). This paper asks: if we give classical models the same level of flexibility, does the neural advantage disappear?
Methods
The authors developed TORCHPP, a modular framework that allows for a "fair fight" by systematically increasing the flexibility of both classical and neural models.
- Flexible Hawkes Processes: Instead of fixed exponential decays, they implemented kernels and baselines using learnable linear splines. They also introduced inhibitive variants using a Softplus clamping function, allowing events to suppress (rather than just excite) future occurrences.
- Neural TPPs: They evaluated standard architectures like THP and SAHP but introduced NeuralSplineTPP, which uses a Transformer encoder paired with a spline-based intensity head. This allows for closed-form cumulative intensity and exact sampling.
- Likelihood Decomposition: A key methodological contribution is decomposing the joint log-likelihood into two terms:
- $\log p(t)$: How well the model predicts the timing of the next event.
- $\log p(d \mid t)$: How well the model predicts the type (mark) of the next event, given the time.
Datasets
The study spans seven datasets, ranging from small benchmarks to massive medical records:
- Standard Benchmarks: Taxi (NYC trips), StackOverflow (user badges), Retweet, Amazon (reviews), and Taobao (user behavior).
- Large-Scale Medical Tasks:
- UK Biobank (UKB): Disease histories for 500k participants (1,257 event types).
- MIMIC-IV: ICU records containing procedures and medications (117 event types).
Key Insights and Ideas
- The "Gap" is an Artifact: Once Hawkes processes are given flexible splines and inhibitive kernels, the performance gap reported in previous literature largely vanishes on standard benchmarks. Most of these datasets are dominated by pairwise interactions that Hawkes processes can capture perfectly well.
- Flexibility = Better Timing, Not Better Typing: The authors found that increasing model flexibility (more spline knots) significantly improves the time prediction term ($\log p(t)$) but has almost zero impact on the type prediction term ($\log p(d \mid t)$). On most benchmarks, models differ far more in when they think an event will happen than what they think it will be.
- Medical Data is Different: The UK Biobank data, despite its size, is well-explained by additive (Hawkes-like) interactions. However, MIMIC-IV was the only dataset where neural models maintained a clear lead in predicting event types. This suggests that ICU data contains complex, non-additive dependencies that finally justify the overhead of a neural TPP.
Key Results
On the five standard benchmarks, the "souped-up" Hawkes variants matched or beat less flexible neural models (like THP or NHP). For example, on the Taxi dataset, the best Hawkes variant reached a joint log-likelihood nearly identical to the best neural model (a difference of only 0.09 nats).
In the medical domain:
- UK Biobank: Flexible Hawkes processes nearly matched the best neural TPP, trailing by only 0.07 nats.
- MIMIC-IV: NeuralSplineTPP opened a significant gap (0.45 nats) over the best Hawkes variant, particularly in predicting events that occur within the same minute as the previous event.
Limitations and Future Work
- Scope of Models: The study focused on intensity-based models and one intensity-free baseline. It did not cover newer flow-based, diffusion-based, or state-space TPPs.
- Task Limitation: The evaluation was strictly on autoregressive next-event prediction. The authors did not test long-horizon simulation or planning.
- Knot Placement: The
NeuralSplineTPPplaces knots based on time elapsed since the last event, whereas Hawkes splines are anchored to every past trigger event. This structural difference might favor neural models for very short-range dependencies. - Future Direction: The authors suggest that future TPP research should move away from "saturated" benchmarks like Retweet and focus on complex, high-dimensional datasets like MIMIC-IV to truly test the limits of neural architectures.
Summary updated