Chickenpox Cases in Hungary: a Benchmark Dataset for Spatiotemporal Signal Processing with Graph Neural Networks

Benedek Rozemberczki, Paul Scherer, Oliver Kiss, Rik Sarkar, Tamas FerenciarXiv 2021 · arXiv

Introduction

Spatiotemporal forecasting—predicting future states of nodes and edges in a graph based on historical data—is a cornerstone of modern machine learning. While Recurrent Graph Neural Networks (GNNs) have shown immense promise in this area, the field suffers from a significant benchmarking bias. Most models are iteratively evaluated on a narrow set of datasets, primarily focused on urban traffic flow and weather patterns. This lack of diversity limits our understanding of how these architectures generalize to domains with fundamentally different statistical properties.

To address this, the "Chickenpox Cases in Hungary" dataset is introduced as a challenging, real-world epidemiological benchmark. It provides a longitudinal look at disease spread, characterized by unique behaviors like zero-inflation and heteroskedasticity that are absent in standard traffic datasets.

The Hungarian Chickenpox Dataset

The core contribution of this work is a multivariate time series detailing weekly reported chickenpox cases across Hungary's 20 spatial units (19 counties and the capital, Budapest) from January 2005 to January 2015. The underlying graph structure consists of 20 vertices and 61 edges, representing physical borders between these regions.

Weekly reported chickenpox cases across 20 Hungarian counties, highlighting the seasonal variations and zero-inflation in smaller regions.

Unlike traffic data, which often follows predictable daily or weekly cycles, epidemiological data is influenced by complex human behaviors and environmental factors. For instance, Budapest has ten times the population of the smallest county, leading to massive scaling differences. Furthermore, transmission drops drastically during summer school holidays, causing many smaller counties to report zero cases for extended periods—a phenomenon known as zero-inflation.

Statistical Challenges and Insights

The dataset presents several hurdles for standard GNN architectures:

  1. Heteroskedasticity: The variance of the time series changes significantly over time, making it difficult for models to maintain consistent predictive accuracy.
  2. Spatial and Temporal Autocorrelation: Cases naturally spread to neighboring counties over time. This spatial dependency is measurable and persistent.
Spatial autocorrelation of differenced case counts measured by Moran's Index, demonstrating the persistent spatial dependencies in the data.
  1. Structural Changes: Over the decade-long span, population shifts and changes in reporting standards introduce non-stationarity into the data.

Benchmarking Methodology

The authors evaluated a suite of state-of-the-art recurrent GNN models using the PyTorch Geometric Temporal library. The models tested include:

  • GConvLSTM and GConvGRU: Standard recurrent units integrated with graph convolutions.
  • Evolve GCN (O and H variants): Models that evolve GCN parameters over time.
  • DynGRAE, STGCN, and DCRNN: Advanced architectures designed for dynamic graph signals.

All models were trained using 8 temporal lags as input features and 32-dimensional graph convolutional filters, optimized over 200 epochs.

Key Results

The models were tested across forecasting horizons of 10, 20, and 40 weeks.

  • Top Performer: The DynGRAE architecture emerged as the most robust, consistently achieving the lowest Mean Squared Error (MSE) across all horizons (e.g., 0.706 at 10 weeks).
  • Model Discrepancies: Significant performance gaps were observed; for example, Evolve GCN-O struggled significantly compared to DynGRAE, suggesting that parameter evolution alone may not capture epidemiological dynamics.
  • Horizon Degradation: As the forecasting window extended to 40 weeks, the performance of several models degraded to the point of being worse than random guessing, highlighting the difficulty of long-term epidemiological prediction.

Conclusion

The Hungarian Chickenpox dataset reveals that current GNN architectures are only "moderately well suited" for epidemiological forecasting. The prevalence of count data and zero-inflation suggests a need for specialized loss functions and output activations that respect the underlying distribution of the data. This benchmark serves as a call to action for the machine learning community to develop more versatile spatiotemporal models capable of handling the messy, non-linear realities of public health data.

Summary updated

Abstract

Recurrent graph convolutional neural networks are highly effective machine learning techniques for spatiotemporal signal processing. Newly proposed graph neural network architectures are repetitively evaluated on standard tasks such as traffic or weather forecasting. In this paper, we propose the Chickenpox Cases in Hungary dataset as a new dataset for comparing graph neural network architectures. Our time series analysis and forecasting experiments demonstrate that the Chickenpox Cases in Hungary dataset is adequate for comparing the predictive performance and forecasting capabilities of novel recurrent graph neural network architectures.
BibTeX
@misc{rozemberczki2021chickenpox,
  author = {Benedek Rozemberczki and Paul Scherer and Oliver Kiss and Rik Sarkar and Tamas Ferenci},
  title = {{Chickenpox Cases in Hungary: a Benchmark Dataset for Spatiotemporal Signal Processing with Graph Neural Networks}},
  year = {2021},
  eprint = {2102.08100},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2102.08100},
}
Figure