Introduction
Spatiotemporal forecasting—predicting future states of nodes and edges in a graph based on historical data—is a cornerstone of modern machine learning. While Recurrent Graph Neural Networks (GNNs) have shown immense promise in this area, the field suffers from a significant benchmarking bias. Most models are iteratively evaluated on a narrow set of datasets, primarily focused on urban traffic flow and weather patterns. This lack of diversity limits our understanding of how these architectures generalize to domains with fundamentally different statistical properties.
To address this, the "Chickenpox Cases in Hungary" dataset is introduced as a challenging, real-world epidemiological benchmark. It provides a longitudinal look at disease spread, characterized by unique behaviors like zero-inflation and heteroskedasticity that are absent in standard traffic datasets.
The Hungarian Chickenpox Dataset
The core contribution of this work is a multivariate time series detailing weekly reported chickenpox cases across Hungary's 20 spatial units (19 counties and the capital, Budapest) from January 2005 to January 2015. The underlying graph structure consists of 20 vertices and 61 edges, representing physical borders between these regions.
Unlike traffic data, which often follows predictable daily or weekly cycles, epidemiological data is influenced by complex human behaviors and environmental factors. For instance, Budapest has ten times the population of the smallest county, leading to massive scaling differences. Furthermore, transmission drops drastically during summer school holidays, causing many smaller counties to report zero cases for extended periods—a phenomenon known as zero-inflation.
Statistical Challenges and Insights
The dataset presents several hurdles for standard GNN architectures:
- Heteroskedasticity: The variance of the time series changes significantly over time, making it difficult for models to maintain consistent predictive accuracy.
- Spatial and Temporal Autocorrelation: Cases naturally spread to neighboring counties over time. This spatial dependency is measurable and persistent.
- Structural Changes: Over the decade-long span, population shifts and changes in reporting standards introduce non-stationarity into the data.
Benchmarking Methodology
The authors evaluated a suite of state-of-the-art recurrent GNN models using the PyTorch Geometric Temporal library. The models tested include:
- GConvLSTM and GConvGRU: Standard recurrent units integrated with graph convolutions.
- Evolve GCN (O and H variants): Models that evolve GCN parameters over time.
- DynGRAE, STGCN, and DCRNN: Advanced architectures designed for dynamic graph signals.
All models were trained using 8 temporal lags as input features and 32-dimensional graph convolutional filters, optimized over 200 epochs.
Key Results
The models were tested across forecasting horizons of 10, 20, and 40 weeks.
- Top Performer: The DynGRAE architecture emerged as the most robust, consistently achieving the lowest Mean Squared Error (MSE) across all horizons (e.g., 0.706 at 10 weeks).
- Model Discrepancies: Significant performance gaps were observed; for example, Evolve GCN-O struggled significantly compared to DynGRAE, suggesting that parameter evolution alone may not capture epidemiological dynamics.
- Horizon Degradation: As the forecasting window extended to 40 weeks, the performance of several models degraded to the point of being worse than random guessing, highlighting the difficulty of long-term epidemiological prediction.
Conclusion
The Hungarian Chickenpox dataset reveals that current GNN architectures are only "moderately well suited" for epidemiological forecasting. The prevalence of count data and zero-inflation suggests a need for specialized loss functions and output activations that respect the underlying distribution of the data. This benchmark serves as a call to action for the machine learning community to develop more versatile spatiotemporal models capable of handling the messy, non-linear realities of public health data.
Summary updated