Surfacing Estimation Uncertainty in the Decay Parameters of Hawkes Processes with Exponential Kernels

Tiago Santos, Florian Lemmerich, Denis HelicarXiv 2021 · OpenReview

Abstract

As a tool for capturing irregular temporal dependencies (rather than resorting to binning temporal observations to construct time series), Hawkes processes with exponential decay have seen widespread adoption across many application domains, such as predicting the occurrence time of the next earthquake or stock market spike. However, practical applications of Hawkes processes face a noteworthy challenge: There is substantial and often unquantified variance in decay parameter estimations, especially in the case of a small number of observations or when the dynamics behind the observed data suddenly change. We empirically study the cause of these practical challenges and we develop an approach to surface and thereby mitigate them. In particular, our inspections of the Hawkes process likelihood function uncover the properties of the uncertainty when fitting the decay parameter. We thus propose to explicitly capture this uncertainty within a Bayesian framework. With a series of experiments with synthetic and real-world data from domains such as "classical" earthquake modeling or the manifestation of collective emotions on Twitter, we demonstrate that our proposed approach helps to quantify uncertainty and thereby to understand and fit Hawkes processes in practice.

Introduction

Hawkes processes are essential for modeling "bursty" event sequences where an initial occurrence triggers a cascade of subsequent events. From seismology to social media dynamics, these processes capture how the influence of an event fades over time. This fading is governed by a critical "decay" parameter, $\beta$. However, estimating $\beta$ accurately is notoriously difficult. Standard point estimation methods often fail because the mathematical landscape of the likelihood function is "noisy and non-convex," leading to unreliable conclusions. This paper proposes a parsimonious Bayesian framework that quantifies the uncertainty of these estimates, providing a more robust foundation for researchers.

The Hawkes Process

A Hawkes process is a "self-exciting" point process: an event increases the probability of future events, which then decay over time back to a baseline. As shown in the intensity diagram below, the intensity function $\lambda(t)$ jumps at event times $t_i$ and decays exponentially according to the parameter $\beta$.

Intensity function of a Hawkes process with exponential decay.

In practice, researchers typically use optimization algorithms to find a single "best" value for $\beta$. However, if the dataset is small or the underlying system dynamics shift (non-stationarity), these point estimates can be wildly inaccurate. Most current research treats these estimates as certain, which can lead to incorrect interpretations of event influence.

The Challenge of Decay Estimation

The core difficulty lies in the nature of the likelihood surface. The authors demonstrate that the negative log-likelihood (NLL) function for $\beta$ is often jagged and lacks a clear, single global minimum.

Negative log-likelihood surfaces for the decay parameter beta across large, medium, and small ranges.

As illustrated in the likelihood surfaces above, zooming in on the "optimal" decay value reveals a "noisy basin" where the true minimum is obscured. This proves that the NLL surface is neither concave nor convex, explaining why standard gradient-based optimizers frequently struggle to find the true parameter value.

Proposed Bayesian Framework

To address these challenges, the authors shift from point estimation to Bayesian inference. Their approach involves several key components:

  1. Likelihood Analysis: By proving the non-convexity of the NLL surface, they justify the need for a probabilistic approach.
  2. Distributional Assumptions: They treat individual realizations of a process as samples, assuming the likelihood of estimated decay values follows an Exponential distribution and utilizing a Gamma distribution as a conjugate prior.
  3. Efficiency: This choice of distributions allows for a closed-form solution for the posterior predictive mean, making the framework computationally efficient without requiring intensive Markov Chain Monte Carlo (MCMC) simulations.
  4. Changepoint Detection: The model is extended to handle "breaks in stationarity"—moments where system dynamics shift suddenly—using a Bayesian changepoint model.

Experimental Validation

The framework was tested across synthetic and real-world datasets, yielding several key insights:

  • Seismology: The model successfully replicated known relationships in Japanese earthquake data, identifying that Hida earthquakes precede Kwanto ones with high certainty ($\beta \approx 31.71$).
  • Cognitive Effort in Learning: Analyzing Duolingo logs, the authors found that "harder" vocabulary words (C-level) exhibit lower decay rates. This suggests that learning bursts for difficult material are more sustained, quantifying increased cognitive effort.
  • Social Shocks: In the wake of the November 2015 Paris terror attacks, the model detected a 15% decrease in the mean posterior decay. This confirms the "collective effervescence" hypothesis: the event didn't just increase tweet volume; it made social media behavior more "bursty" and sustained.

Conclusion

While the proposed Bayesian approach provides a robust method for surfacing uncertainty, it relies on specific distributional assumptions for the decay estimates. Future research may explore more complex functional kernels and advanced optimization techniques, such as disciplined convex-concave programming, to further refine how we navigate the complex likelihood landscapes of self-exciting processes.

Summary updated

Abstract

As a tool for capturing irregular temporal dependencies (rather than resorting to binning temporal observations to construct time series), Hawkes processes with exponential decay have seen widespread adoption across many application domains, such as predicting the occurrence time of the next earthquake or stock market spike. However, practical applications of Hawkes processes face a noteworthy challenge: There is substantial and often unquantified variance in decay parameter estimations, especially in the case of a small number of observations or when the dynamics behind the observed data suddenly change. We empirically study the cause of these practical challenges and we develop an approach to surface and thereby mitigate them. In particular, our inspections of the Hawkes process likelihood function uncover the properties of the uncertainty when fitting the decay parameter. We thus propose to explicitly capture this uncertainty within a Bayesian framework. With a series of experiments with synthetic and real-world data from domains such as "classical" earthquake modeling or the manifestation of collective emotions on Twitter, we demonstrate that our proposed approach helps to quantify uncertainty and thereby to understand and fit Hawkes processes in practice.
BibTeX
@misc{santos2021surfacing,
  author = {Tiago Santos and Florian Lemmerich and Denis Helic},
  title = {{Surfacing Estimation Uncertainty in the Decay Parameters of Hawkes Processes with Exponential Kernels}},
  year = {2021},
  eprint = {2104.01029},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2104.01029},
}
Figure