Pandemic Control, Game Theory and Machine Learning

Yao Xuan, Robert Balkin, Jiequn Han, Ruimeng Hu, Héctor D. Ceniceros

Abstract

This paper introduces a multi-region stochastic framework for pandemic control that models neighboring regions as strategic players in a non-cooperative game. By integrating epidemiological SEIR models with a novel deep learning algorithm, Enhanced Deep Fictitious Play, the authors efficiently solve for Nash Equilibria in high-dimensional systems. This approach provides a roadmap for decentralized authorities to balance public health outcomes with economic stability in an interconnected society.

Introduction

Pandemic management is a complex balancing act: aggressive lockdowns save lives but devastate the economy, while inaction leads to catastrophic public health crises. This challenge is compounded by regional interconnectivity. In a highly mobile society, a lockdown in one state directly impacts the infection rates and economic flow of its neighbors. Traditional epidemiological models often struggle with this "strategic" nature of decentralized governance and the "curse of dimensionality" inherent in modeling multiple stochastic regions.

The Multi-Region SEIR Framework

The authors propose a multi-region stochastic SEIR (Susceptible, Exposed, Infectious, Removed) model. In this framework, each region's planner seeks to minimize a cost function that includes economic losses from lockdowns, the "value of life" lost to mortality, and healthcare expenses.

The standard SEIR epidemiological model serves as the foundation for the proposed framework.

Unlike isolated models, this framework acknowledges that regions are not islands. Travel between states creates feedback loops where one state's policy can "leak" infections into a neighbor. This necessitates a game-theoretic approach where each region acts in its own interest while anticipating the rational moves of its neighbors.

Solving the Game: Enhanced Deep Fictitious Play

To solve for the Nash Equilibrium in these high-dimensional spaces, the researchers developed Enhanced Deep Fictitious Play (DFP). This algorithm addresses the computational exhaustion typical of standard numerical methods when dealing with stochastic differential equations.

The multi-region stochastic SEIR model captures strategic interdependence and infection 'leakage' between neighboring regions.

Key technical innovations include:

  • Deep Learning Decoupling: Instead of solving a single coupled system, the algorithm approximates each region's optimal policy using neural networks—specifically, V-networks for value functions and $\alpha$-networks for policies.
  • BSDE Transformation: The researchers transform complex Partial Differential Equations (PDEs) into Backward Stochastic Differential Equations (BSDEs), which neural networks are particularly adept at solving.
  • Computational Efficiency: Standard fictitious play is memory-intensive. Enhanced DFP uses $\alpha$-networks to "proxy" the optimal strategies of other players, reducing time complexity from $O(M^2)$ to $O(M)$ and memory from $O(M)$ to $O(1)$.

Case Study: The NY-NJ-PA Corridor

The model was calibrated using COVID-19 parameters and demographic data for New York, New Jersey, and Pennsylvania. A 180-day simulation revealed several key insights:

  • Proactive Lockdown: The Nash Equilibrium strategy suggests implementing strict lockdown policies early in the pandemic and gradually relaxing them as infections subside.
  • Effective Mitigation: Under this equilibrium, the percentage of the population in the "Infectious" and "Exposed" categories remains low and stable, preventing the spikes seen in scenarios where governors remain inactive.
  • Scalability: The Enhanced DFP algorithm successfully handled the $3N+1$ dimensional state space, proving it can scale to complex problems that would break traditional solvers.

Conclusion and Future Directions

While the model provides a realistic perspective on decentralized pandemic management, it has limitations. Current assumptions include a closed system and homogeneous populations. Future iterations should incorporate age-stratified data, asymptomatic compartments, and more nuanced vaccination dynamics (accounting for waning immunity or breakthrough infections) to further increase accuracy.

Summary updated

Abstract

Game theory has been an effective tool in the control of disease spread and in suggesting optimal policies at both individual and area levels. In this AMS Notices article, we focus on the decision-making development for the intervention of COVID-19, aiming to provide mathematical models and efficient machine learning methods, and justifications for related policies that have been implemented in the past and explain how the authorities' decisions affect their neighboring regions from a game theory viewpoint.
BibTeX
@misc{xuan2022pandemic,
  author = {Yao Xuan and Robert Balkin and Jiequn Han and Ruimeng Hu and Héctor D. Ceniceros},
  title = {{Pandemic Control, Game Theory and Machine Learning}},
  year = {2022},
  eprint = {2208.08646},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2208.08646},
}
Figure