Introduction
Deep learning has revolutionized spatiotemporal forecasting, yet most models provide only point estimates. In high-stakes domains like epidemic tracking or traffic management, a single "best guess" can be misleading without an accompanying measure of uncertainty. This paper provides the first systematic benchmark of uncertainty quantification (UQ) methods for spatiotemporal data, evaluating the performance of Bayesian versus Frequentist approaches across grid-based and graph-based architectures.
Spatiotemporal data presents unique challenges for UQ, including complex geographical and historical correlations, error propagation over long horizons, and significant computational overhead. This study addresses these by casting various UQ methods into a unified framework based on Statistical Decision Theory.
Methodology
The researchers evaluated six primary UQ methods integrated with base architectures such as ConvLSTM (for grid data) and DCRNN (for graph data). These methods include:
- Frequentist Approaches: Bootstrap (retraining on resampled data), Quantile Regression (predicting percentiles), Spline Quantile (preventing quantile crossing), and MIS Regression (optimizing for the Mean Interval Score).
- Bayesian Approaches: MC Dropout (inference-time dropout) and SG-MCMC (sampling from the weight posterior using Stochastic Gradient Markov Chain Monte Carlo).
Experimental Setup
The benchmark utilized three diverse real-world datasets:
- Air Quality (PM2.5): Hourly grid data from 35 stations in Beijing.
- METR-LA Traffic: Graph-based vehicle speed data from 207 sensors in Los Angeles.
- COVID-19 Incident Deaths: Weekly death counts across 50 US states, utilizing air traffic data to define spatial dependencies.
For the COVID-19 dataset, the authors introduced DeepGLEAM, a hybrid model that combines mechanistic epidemic modeling with deep learning to improve performance when data is sparse.
Key Insights and Results
The study identifies a critical trade-off between accuracy and coverage. Bayesian methods, specifically SG-MCMC, proved to be the champions of Mean Absolute Error (MAE), providing the most accurate average predictions. In contrast, Frequentist methods like MIS Regression yielded the best Mean Interval Scores (MIS), offering more reliable confidence intervals for risk assessment.
Sample Efficiency
A key finding was that SG-MCMC requires significantly fewer samples to reach a stable uncertainty estimate compared to Bootstrapping. This suggests that posterior contraction allows Bayesian models to converge on uncertainty more efficiently than simple data resampling.
Overconfidence and Hybrid Modeling
The benchmark confirms that deep learning models are prone to overconfidence, often producing narrow intervals that miss true values, particularly during distribution shifts. The hybrid DeepGLEAM approach significantly outperformed pure deep learning models by incorporating "physics-informed" inductive biases.
Conclusion
While SG-MCMC and MIS Regression offer superior accuracy and uncertainty reliability respectively, they remain computationally expensive. Future research should focus on baking spatiotemporal graph/grid structures directly into the inference process and improving the scalability of these methods for real-time applications.
Summary updated