Pyfectious: An individual-level simulator to discover optimal containment polices for epidemic diseases

Arash Mehrjou, Ashkan Soleymani, Amin Abyaneh, Bernhard Schölkopf, Stefan Bauer

TL;DR

This paper introduces "Pyfectious," a novel individual-level simulator for epidemic diseases that allows for highly granular policy testing and discovery. By modeling populations probabilistically and simulating at the individual level, Pyfectious enables the development of more effective, nuanced containment strategies than traditional compartment models, crucial for managing real-world outbreaks like COVID-19.

What Problem Does This Paper Address?

Controlling infectious disease outbreaks relies heavily on understanding how diseases spread and evaluating the effectiveness of various containment policies. Traditional epidemiological models, like compartment models (e.g., SIR, SEIR), simplify populations into broad categories, which limits their ability to assess the impact of "intelligent" policies that target specific individuals or groups. This lack of granularity makes it difficult to study nuanced interventions like identifying and isolating superspreaders or tailoring policies to individual behaviors. The authors aim to bridge this gap by developing a simulator that models each person individually, allowing for a more detailed and realistic exploration of epidemic dynamics and policy effectiveness.

Methods

Pyfectious is an individual-level simulator built on a probabilistic generative model of a structured population. Key methodological contributions include:

  • Individual-Level Simulation: Unlike compartment models, Pyfectious models each person as an independent agent with attributes (age, health, etc.) and a daily schedule.
  • Probabilistic Population Generation: The entire population is treated as a hierarchical random variable, meaning parameters are sampled from distributions. This allows for robust conclusions and confidence bounds, as slight variations in input don't drastically alter outcomes.
  • Clock/Event-Based Simulation: A hybrid approach combines event-driven simulation with a background clock to manage temporal dynamics efficiently. This allows for multiresolution time modeling.
  • Flexible Policy Enforcement: Pyfectious supports a wide range of policies, from broad lockdowns to individual quarantines, and even probabilistic or conditional policies. It's designed to be a reinforcement learning environment.
  • Connectivity Graph: Interactions are modeled using a graph structure that captures relationships within families, communities, and subcommunities.

Datasets

The paper uses real-world data to parameterize the simulator for COVID-19. Specifically, it references:

  • United States Census Bureau data for population structure.
  • Eurostat data for demographics of European countries.
  • Data from Wuhan (China) for initial infection rate parameters.
  • Various studies and reports for disease parameters like incubation period, disease period, and death probability.

No new datasets were introduced; rather, existing public data was used to inform the simulator's configuration.

Key Insights and Ideas

  • Granularity Matters: Simulating at the individual level unlocks the potential for discovering highly specific and effective control policies that are impossible with aggregate models.
  • Probabilistic Modeling Enhances Robustness: Treating the population and its attributes as random variables provides confidence intervals for simulation results, making decisions more reliable.
  • Flexibility is Key: Pyfectious's design allows for a wide array of policy types (universal, partial, conditional, probabilistic) and population structures, making it adaptable to various scenarios.
  • Reinforcement Learning Potential: The simulator is explicitly designed as an environment for RL agents to learn optimal control strategies, a powerful approach for complex, dynamic problems like pandemics.
  • Counter-Intuitive Policies Can Emerge: The optimization experiments suggest that the most effective policies might not always be the most intuitive, highlighting the value of automated discovery.

Key Results

  • Scalability: The simulator demonstrates linear scaling with population size for model generation and simulation time, suggesting it can handle large populations.
  • Performance: Experiments show Pyfectious can simulate 48 hours for a city of 20,000 individuals in a reasonable time frame.
  • Policy Effectiveness: Simulations demonstrate that earlier and more targeted interventions (e.g., quarantining specific roles or sectors) can significantly reduce the peak number of active cases and overall infections compared to no intervention or broad, late-stage measures.
  • Optimal Policy Discovery: An optimization experiment identified a policy that restricted 60% of students and workers, and 20% of customers, which was found to be effective in minimizing active cases while respecting economic constraints. This policy was found to be more effective than simply restricting all groups maximally.

Limitations and Future Work

  • Computational Cost: While designed for efficiency, complex simulations with large populations and fine-grained policies can still be computationally intensive.
  • Data Dependency: The accuracy of the simulations is inherently dependent on the quality and availability of real-world data for parameterization.
  • Future Work: The authors plan to explore general-purpose reinforcement learning algorithms, particularly those combining representation learning with Monte Carlo Tree Search (like AlphaZero), to further enhance the discovery of complex, human-expert-inaccessible policies. They also aim to adapt the simulator for various population sizes and disease structures.

Summary updated

Abstract

Simulating the spread of infectious diseases in human communities is critical for predicting the trajectory of an epidemic and verifying various policies to control the devastating impacts of the outbreak. Many existing simulators are based on compartment models that divide people into a few subsets and simulate the dynamics among those subsets using hypothesized differential equations. However, these models lack the requisite granularity to study the effect of intelligent policies that influence every individual in a particular way. In this work, we introduce a simulator software capable of modeling a population structure and controlling the disease's propagation at an individualistic level. In order to estimate the confidence of the conclusions drawn from the simulator, we employ a comprehensive probabilistic approach where the entire population is constructed as a hierarchical random variable. This approach makes the inferred conclusions more robust against sampling artifacts and gives confidence bounds for decisions based on the simulation results. To showcase potential applications, the simulator parameters are set based on the formal statistics of the COVID-19 pandemic, and the outcome of a wide range of control measures is investigated. Furthermore, the simulator is used as the environment of a reinforcement learning problem to find the optimal policies to control the pandemic. The obtained experimental results indicate the simulator's adaptability and capacity in making sound predictions and a successful policy derivation example based on real-world data. As an exemplary application, our results show that the proposed policy discovery method can lead to control measures that produce significantly fewer infected individuals in the population and protect the health system against saturation.
BibTeX
@article{mehrjou2023pyfectious,
  author = {Arash Mehrjou and Ashkan Soleymani and Amin Abyaneh and Bernhard Schölkopf and Stefan Bauer},
  title = {{Pyfectious: An individual-level simulator to discover optimal containment polices for epidemic diseases}},
  journal = {PLOS Computational Biology},
  year = {2023},
  doi = {10.1371/journal.pcbi.1010799},
}
Figure