Why Can’t Neural Networks Forecast Pandemics Better

Joshua D. Zelek, John Zelek, Alexander WongJournal of Computational Vision and Imaging Systems 2021 · DOI

Abstract

Why can’t neural networks (NN) forecast better? In major super-forecasting competitions, NNs have typically under-performed compared to traditional statistical methods. When they have performed well, the underlying methods have been ensembles of NN and statistical methods. In this study, we evaluate the strengths of a collection of methods for forecasting pandemics such as COVID-19 using NNs, statistical methods, and parameterized mechanistic models. We show that NNs typically under-perform in forecasting COVID-19 active cases, which can be attributed to the lack of training data common for such forecasts. Our test data consists of the top ten countries for active COVID-19 cases early in the pandemic. We found that statistical methods outperform NNs in most cases, though NNs remain powerful pattern finders that may be better utilized in hybrid models.

The Paradox of Pandemic Forecasting

In the world of time-series (TS) forecasting, a persistent paradox exists: despite their massive success in computer vision and natural language processing, Neural Networks often lose to traditional statistical methods in major competitions. This is particularly evident in pandemic modeling. Pandemic data is notoriously difficult: it is often "short" (limited historical data), geographically localized, and subject to noisy reporting. This study investigates why NNs—essentially high-powered pattern finders—fail to provide superior forecasts for active COVID-19 cases compared to models developed decades ago.

Methodology and Data

The researchers compared three distinct modeling philosophies:

  1. Mechanistic Models: Causal models based on epidemiology, such as SIR and SEIR, using differential equations.
  2. Statistical Models: Data-driven classical methods including ARIMA, Holt-Winters (HWAAS), TBAT, and Facebook’s Prophet.
  3. Neural Network Models: State-of-the-art architectures like N-BEATS (residual links) and DeepAR (probabilistic RNNs).

The study utilized a Kaggle dataset of COVID-19 cases from early 2020, focusing on the top ten countries by total confirmed cases. The data was formatted as time series representing the percentage of active cases relative to the total population, using 72 instances for training and a 7-day window for prediction.

Key Results: Statistical Dominance

The quantitative results revealed a clear advantage for "old school" mathematics. Statistical methods (ARIMA, TBAT, HWAAS) outperformed deep learning models in 8 out of 10 countries analyzed.

Table 1: Comparative error metrics (RMSE) across models and countries. Bold values indicate the best performing model for each country.

As shown in the table above, ARIMA was the most robust performer for the US, Italy, and Russia, while TBAT led in four other nations. Neural Networks (N-Beats and DeepAR) only took the lead in France and Brazil, respectively.

Visualizing the Performance Gap

Qualitative analysis of the forecast plots highlighted why NNs struggle. While statistical models generally followed the trajectory of the infection curves, NNs often produced "flat" or overly simplistic forecasts when faced with limited data.

Figure 1: US predictions for early May 2020. Note how some models, particularly NBEATS (purple), produce relatively flat trajectories compared to the target and statistical benchmarks.

Why NNs Struggle

  • The "Data Hunger" Problem: NNs require massive amounts of data to estimate millions of parameters. Pandemic time series are often too short and lack the high-frequency data where NNs typically shine.
  • The Random Walk Trap: NNs often perform well on data that mimics a "random walk." In countries like Brazil and France, where the infection curve followed such a pattern, NNs were more competitive.
  • Domain Expertise: Statistical and mechanistic models incorporate "prior knowledge" about trend behavior. Purely data-driven NNs start from scratch, making them significantly less sample-efficient in data-sparse environments.

Conclusion and Future Work

The authors suggest that NNs should not be used in isolation for pandemic forecasting. The path forward lies in hybridization—specifically Physics-Informed Neural Networks (PINNs). By embedding epidemiological differential equations (like SIR) directly into the loss function of a network, researchers can force the AI to respect the laws of biology while still benefiting from the pattern-matching power of deep learning.

Summary updated

Abstract


 Why can’t neural networks (NN) forecast better? In the major super-forecasting competitions, NN have typically under-performed when compared to traditional statistical methods. When they have performed well, the underlying methods have been ensembles of NN and statistical methods. Forecasting stock markets, medical, infrastructure dynamics, social activity or pandemics each have their own challenges. In this study, we evaluate the strengths of a collection of methods for forecasting pandemics such as Covid-19 using NN, statistical methods as well as parameterized mechanistic models. Forecasts of epidemics can inform public health response and decision making, so accurate forecasting is crucial for general public notification, timing and spatial targeting of intervention. We show that NN typically under-perform in forecasting Covid-19 active cases which can be attributed to the lack of training data which is common for forecasts. Our test data consists of the top ten countries for active Covid-19 cases early in the pandemic and is represented as a Time Series (TS). We found that Statistical methods outperform NN for most cases. Albeit, NN are still good pattern finders and we suggest that there are perhaps more productive ways other than purely
 data driven models of using NN to help produce better forecasts.
 
BibTeX
@article{zelek2021why,
  author = {Joshua D. Zelek and John Zelek and Alexander Wong},
  title = {{Why Can’t Neural Networks Forecast Pandemics Better}},
  journal = {Journal of Computational Vision and Imaging Systems},
  year = {2021},
  doi = {10.15353/jcvis.v6i1.3551},
}
Figure