TL;DR
This paper introduces NEST, a novel neural network-based model for understanding and predicting spatio-temporal event data (like earthquakes or crime incidents). NEST uses flexible Gaussian kernels to capture complex spatial patterns and is trained efficiently using imitation learning, outperforming existing methods.
What Problem Does This Paper Address?
Many real-world phenomena generate discrete events that occur at specific times and locations. Examples include earthquakes, crime reports, social media posts, and disease outbreaks. Understanding the patterns and predicting future events in such spatio-temporal data is crucial for various applications. While significant progress has been made in modeling temporal event sequences, capturing the intricate spatial dependencies and their interplay with time has remained a challenge. Existing models often rely on strong, simplified assumptions about spatial influence (e.g., isotropic decay) that fail to capture complex, heterogeneous patterns like those influenced by geological fault lines or urban infrastructure. This paper aims to address this gap by proposing a more flexible and interpretable model for spatio-temporal event data.
Methods
The core of the proposed solution is the Neural Embedding Spatio-Temporal (NEST) point process model. NEST addresses the limitations of traditional models by:
- Heterogeneous Gaussian Diffusion Kernels: Instead of simple, location-invariant kernels, NEST uses a mixture of generalized Gaussian diffusion kernels. The parameters of these kernels (shape, orientation, and intensity) are allowed to vary continuously across space. This allows NEST to capture complex, anisotropic spatial influences.
- Neural Network Parameterization: The spatial variation of these kernel parameters is learned by neural networks. This provides flexibility in modeling intricate spatial dependencies without resorting to a fully black-box neural network for the entire process. The neural network learns a latent embedding of spatial locations, which then dictates the kernel parameters. This approach balances flexibility with interpretability, as the learned kernel shapes can be visualized.
- Imitation Learning for Model Fitting: The authors propose an efficient imitation learning (IL) approach for training NEST. Unlike traditional Maximum Likelihood Estimation (MLE), IL directly measures the divergence between the empirical distribution of real data and the distribution of data generated by the model. This makes the IL approach more robust to model misspecification and less reliant on strict parametric assumptions. The IL framework is framed as a reinforcement learning problem where a "learner" policy tries to imitate an "expert" (the real data). A key innovation is the use of a data-driven, closed-form reward function based on Maximum Mean Discrepancy (MMD), which avoids the need for complex inverse reinforcement learning.
Datasets
The paper evaluates NEST on two real-world datasets:
- Atlanta 911 calls-for-services data: Specifically, robbery reports from 2015-2017 were used to model crime event sequences.
- Northern California seismic data: Earthquake records from 1978-2018 were used to model seismic activity.
The authors also generated synthetic datasets to validate the model's ability to capture underlying spatial information, including both linearly and non-linearly dependent spatial parameters.
Key Insights and Ideas
- Spatial Heterogeneity is Key: The paper strongly emphasizes that real-world spatio-temporal events often exhibit complex, non-uniform spatial influences (e.g., along fault lines or roads) that simpler models miss. NEST's ability to model this heterogeneity is a major contribution.
- Interpretability through Kernels: By parameterizing the spatial influence with Gaussian kernels, NEST offers a degree of interpretability. Visualizing the learned kernel parameters can reveal insights into how events influence their surroundings, which is valuable for domain experts.
- Imitation Learning for Robustness: The adoption of imitation learning for model fitting is a significant methodological choice. It allows the model to learn from data without being overly constrained by potentially incorrect parametric assumptions, leading to more robust performance.
- Efficient Training: The proposed imitation learning framework, combined with an efficient thinning algorithm for sampling, makes training NEST computationally feasible, even for complex spatio-temporal data.
Key Results
- Superior Performance: NEST, both with imitation learning (NEST+IL) and MLE (NEST+MLE), consistently outperformed baseline methods like ETAS and RLPP on both synthetic and real-world datasets. This was measured by average Mean Squared Error (MSE) for one-step-ahead prediction and average absolute Maximum Mean Discrepancy (MMD).
- Improved Log-Likelihood: NEST achieved higher log-likelihood values on real-world datasets compared to ETAS, indicating a better fit to the data.
- Capturing Complex Patterns: Visualizations of the conditional intensity from NEST showed a much finer and more nuanced capture of spatial influence compared to ETAS, especially in scenarios with complex spatial structures (e.g., along fault lines or urban road networks).
- Effect of Gaussian Components: Increasing the number of Gaussian mixture components (K) generally improved performance, with diminishing returns observed beyond K=5 for the tested datasets.
Limitations and Future Work
- Computational Cost: While the IL approach is efficient, generating samples for training can still be time-consuming, especially with a large number of Gaussian components. The paper notes that NEST+MLE is generally faster than NEST+IL.
- Marked Processes: The current NEST model focuses on the time and location of events. The authors suggest extending the model to handle "marked" spatio-temporal processes, where events have additional attributes (e.g., earthquake magnitude), as a future direction.
- Incorporating Prior Spatial Information: The paper mentions that incorporating known structural information (like fault maps) into the model could be a valuable future extension, potentially leading to even more accurate and interpretable results.
Summary updated