Negative-Binomial Randomized Gamma Dynamical Systems for Heterogeneous Overdispersed Count Time Sequences

Rui Huang, Sikun Yang, Heinz KoepplIJCAI 2024

Introduction

Modeling count-based time series—such as daily disease mortality rates, word frequencies in news cycles, or user interaction counts—presents unique challenges. Traditional models often struggle with the inherent "burstiness" and extreme variability found in real-world data. Presented at IJCAI-24, this research introduces Negative-Binomial Randomized Gamma Dynamical Systems (NBRGDS), a framework designed to handle heterogeneous overdispersion in count sequences.

The Challenge: Heterogeneous Overdispersion

Most real-world count data is "overdispersed," meaning the variance significantly exceeds the mean. While Poisson-based models like Poisson Gamma Dynamical Systems (PGDS) are improvements over Gaussian models, they often assume a fixed relationship between mean and variance. In reality, different latent trends within the same dataset may exhibit vastly different levels of volatility. NBRGDS addresses this by providing the flexibility to model these varying scales of volatility simultaneously.

Methodology

The core of the proposed framework is the Negative-Binomial Randomized Gamma Markov Process (NBRGMP). This process utilizes a Negative-Binomial distribution to link latent states across time steps, allowing for independent control over the mean and variance.

Graphical representation of the NBRGDS model architecture, illustrating the hierarchical dependencies and the distinction between discrete and continuous latent variables.

The model incorporates two primary structural innovations for the transition kernel:

  1. Factor-structured transitions: Decomposing complex rules into simpler latent components.
  2. Graph-structured transitions: Utilizing a relational gamma process to learn a sparse binary mask, effectively mapping the "social network" of latent factors to show which trends influence others.

Controlling Volatility with $\psi$

A key innovation is the introduction of a specific parameter, $\psi$, which acts as a "magic knob" for the model. By adjusting $\psi$, the system can adapt to different levels of overdispersion and temporal trends.

Simulated count sequences across different values of $\psi$. The parameter allows the model to capture a wide range of behaviors, from explosive growth to rapid decay and stable fluctuations.

As shown in the simulations, the model can represent various dynamics:

  • $\psi < 1$: Captures growing sequences.
  • $\psi = 1$: Represents stable but highly variable sequences.
  • $\psi > 1$: Models decaying sequences with varying degrees of "burstiness."

Experimental Results

The authors evaluated NBRGDS against several baselines using diverse datasets, including political interactions (ICEWS), music listening habits (Last.fm), earthquake reports (EQDB), and COVID-19 mortality data.

  • Accuracy: NBRGDS consistently achieved the lowest Mean Absolute Error (MAE) and Mean Relative Error (MRE) for both data smoothing and forecasting tasks.
  • Efficiency: Despite the increased complexity of the Negative-Binomial distribution, the model demonstrated faster convergence, reaching steady states in significantly fewer iterations than traditional Poisson models.
  • Explainability: The graph-structured version successfully identified distinct "communities" of latent processes, making the underlying dynamics of complex systems like international relations more transparent.

Conclusion

NBRGDS provides a robust and flexible framework for modeling complex count time sequences. By explicitly accounting for heterogeneous overdispersion and learning sparse relational structures, it offers superior predictive performance and enhanced interpretability. Future work may explore time-varying graph structures to capture evolving relationships between latent factors.

Summary updated

BibTeX
@inproceedings{huang2024negative,
  author = {Rui Huang and Sikun Yang and Heinz Koeppl},
  title = {{Negative-Binomial Randomized Gamma Dynamical Systems for Heterogeneous Overdispersed Count Time Sequences}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2024},
}
Figure