Introduction
Modeling count-based time series—such as daily disease mortality rates, word frequencies in news cycles, or user interaction counts—presents unique challenges. Traditional models often struggle with the inherent "burstiness" and extreme variability found in real-world data. Presented at IJCAI-24, this research introduces Negative-Binomial Randomized Gamma Dynamical Systems (NBRGDS), a framework designed to handle heterogeneous overdispersion in count sequences.
The Challenge: Heterogeneous Overdispersion
Most real-world count data is "overdispersed," meaning the variance significantly exceeds the mean. While Poisson-based models like Poisson Gamma Dynamical Systems (PGDS) are improvements over Gaussian models, they often assume a fixed relationship between mean and variance. In reality, different latent trends within the same dataset may exhibit vastly different levels of volatility. NBRGDS addresses this by providing the flexibility to model these varying scales of volatility simultaneously.
Methodology
The core of the proposed framework is the Negative-Binomial Randomized Gamma Markov Process (NBRGMP). This process utilizes a Negative-Binomial distribution to link latent states across time steps, allowing for independent control over the mean and variance.
The model incorporates two primary structural innovations for the transition kernel:
- Factor-structured transitions: Decomposing complex rules into simpler latent components.
- Graph-structured transitions: Utilizing a relational gamma process to learn a sparse binary mask, effectively mapping the "social network" of latent factors to show which trends influence others.
Controlling Volatility with $\psi$
A key innovation is the introduction of a specific parameter, $\psi$, which acts as a "magic knob" for the model. By adjusting $\psi$, the system can adapt to different levels of overdispersion and temporal trends.
As shown in the simulations, the model can represent various dynamics:
- $\psi < 1$: Captures growing sequences.
- $\psi = 1$: Represents stable but highly variable sequences.
- $\psi > 1$: Models decaying sequences with varying degrees of "burstiness."
Experimental Results
The authors evaluated NBRGDS against several baselines using diverse datasets, including political interactions (ICEWS), music listening habits (Last.fm), earthquake reports (EQDB), and COVID-19 mortality data.
- Accuracy: NBRGDS consistently achieved the lowest Mean Absolute Error (MAE) and Mean Relative Error (MRE) for both data smoothing and forecasting tasks.
- Efficiency: Despite the increased complexity of the Negative-Binomial distribution, the model demonstrated faster convergence, reaching steady states in significantly fewer iterations than traditional Poisson models.
- Explainability: The graph-structured version successfully identified distinct "communities" of latent processes, making the underlying dynamics of complex systems like international relations more transparent.
Conclusion
NBRGDS provides a robust and flexible framework for modeling complex count time sequences. By explicitly accounting for heterogeneous overdispersion and learning sparse relational structures, it offers superior predictive performance and enhanced interpretability. Future work may explore time-varying graph structures to capture evolving relationships between latent factors.
Summary updated