Pyfectious: An individual-level simulator to discover optimal containment policies for epidemic diseases

Arash Mehrjou, Ashkan Soleymani, Amin Abyaneh, Samir Bhatt, Bernhard Schölkopf, Stefan Bauer

TL;DR

This paper introduces Pyfectious, a novel individual-level simulator for epidemic diseases that allows for highly granular modeling of populations and disease spread. It's designed to be a flexible environment for reinforcement learning agents to discover complex, optimal containment policies that are difficult for humans to devise, with applications for real-world pandemic response.

What Problem Does This Paper Address?

Predicting epidemic trajectories and evaluating containment policies is crucial for public health. Traditional epidemiological models often rely on compartment models, which simplify populations into broad groups. While computationally efficient, these models lack the granularity to assess the impact of nuanced, individual-focused interventions. This limits our ability to discover and implement highly effective, tailored policies. The authors aim to bridge this gap by creating a simulator that models each individual, their attributes, and their interactions, enabling the exploration of sophisticated control strategies.

Methods

Pyfectious is built around three core components:

  • Population Generator: This module creates a detailed, hierarchical representation of a city's population. It probabilistically assigns attributes (age, gender, health) to individuals, groups them into families and communities, and defines their daily schedules and interaction patterns. A key novelty is its probabilistic generative model, allowing for population synthesis even with incomplete data.
  • Propagation Model: This component simulates the spread of a disease based on individual attributes, disease properties (infection rate, immunity, incubation period), and the connectivity graph. It uses a novel clock/event-based simulation approach that balances computational efficiency with temporal resolution.
  • Simulation Engine: This orchestrates the population and propagation models over time. It incorporates "Command" and "Observer" objects to implement and monitor control policies, respectively. The system is designed to be highly flexible, allowing users to define custom conditions and commands to mimic real-world interventions.

A significant technical contribution is the multi-resolution simulation capability, allowing users to trade off computational resources with temporal accuracy by adjusting the clock period. The authors also emphasize the probabilistic nature of the simulator, where parameters are sampled from distributions, leading to more robust conclusions and confidence bounds.

Datasets

The paper uses real-world data to inform the configuration of the simulator for experiments. Specifically:

  • Population Data: Information from the United States Census Bureau and Eurostat was used to inform the demographic structure of the simulated population.
  • Disease Data: Parameters for COVID-19 (incubation period, disease period, death probability, etc.) were derived from published studies and real-world statistics (e.g., from Wuhan, China, and CDC data).

No new benchmark datasets were introduced; rather, existing data was used to configure the simulator for realistic scenarios.

Key Insights and Ideas

  • Individual-level detail matters: The simulator demonstrates that fine-grained modeling of individual behaviors and interactions is essential for understanding the nuanced effects of control policies.
  • Probabilistic generation is powerful: Creating populations and disease parameters probabilistically leads to more robust simulation results and allows for the estimation of confidence intervals, crucial for decision-making.
  • Reinforcement learning is a natural fit: The simulator's design as a flexible environment with a rich action space makes it ideal for training RL agents to discover complex, non-intuitive control strategies.
  • Counter-intuitive optimal policies can emerge: The optimization experiments revealed that the most effective policies might not always be the most obvious ones, highlighting the value of automated discovery. For instance, a balanced restriction on students and workers, rather than maximum restriction, proved optimal.

Key Results

  • Scalability: Pyfectious demonstrates good scalability, with simulation time growing linearly with population size, making it suitable for simulating large cities.
  • Multi-resolution effectiveness: The simulator can achieve significant speedups (e.g., 4x) by adjusting temporal resolution without drastically altering results, provided the resolution isn't too low.
  • Policy impact: Experiments showed that:
  • Early and well-timed full quarantines are highly effective.
  • Partial quarantines remain effective even with lower detection rates.
  • Targeting specific sectors (like schools and workplaces) can be more effective than broad restrictions on public transport.
  • An optimized policy found by Bayesian optimization recommended restricting 60% of students and workers, and 20% of customers, to minimize active cases while considering economic constraints.

Limitations and Future Work

  • Computational Cost: While scalable, simulating large populations with high temporal resolution can still be computationally demanding.
  • Model Simplifications: The paper acknowledges that real-world scenarios are more complex, and the model makes certain assumptions (e.g., regarding perfect detection in some quarantine scenarios).
  • Future Directions: The authors suggest integrating more sophisticated policy search algorithms, such as those combining representation learning with Monte Carlo Tree Search (like AlphaZero), to discover policies that are even more complex and potentially beyond human intuition. They also envision building upon Pyfectious for more sophisticated policy search algorithms.

Summary updated

Abstract

Simulating the spread of infectious diseases in human communities is critical for predicting the trajectory of an epidemic and verifying various policies to control the devastating impacts of the outbreak. Many existing simulators are based on compartment models that divide people into a few subsets and simulate the dynamics among those subsets using hypothesized differential equations. However, these models lack the requisite granularity to study the effect of intelligent policies that influence every individual in a particular way. In this work, we introduce a simulator software capable of modeling a population structure and controlling the disease’s propagation at an individualistic level. In order to estimate the confidence of the conclusions drawn from the simulator, we employ a comprehensive probabilistic approach where the entire population is constructed as a hierarchical random variable. This approach makes the inferred conclusions more robust against sampling artifacts and gives confidence bounds for decisions based on the simulation results. To showcase potential applications, the simulator parameters are set based on the formal statistics of the COVID-19 pandemic, and the outcome of a wide range of control measures is investigated. Furthermore, the simulator is used as the environment of a reinforcement learning problem to find the optimal policies to control the pandemic. The obtained experimental results indicate the simulator’s adaptability and capacity in making sound predictions and a successful policy derivation example based on real-world data. As an exemplary application, our results show that the proposed policy discovery method can lead to control measures that produce significantly fewer infected individuals in the population and protect the health system against saturation.
BibTeX
@article{mehrjou2023pyfectious,
  author = {Arash Mehrjou and Ashkan Soleymani and Amin Abyaneh and Samir Bhatt and Bernhard Schölkopf and Stefan Bauer},
  title = {{Pyfectious: An individual-level simulator to discover optimal containment policies for epidemic diseases}},
  journal = {PLOS Computational Biology},
  year = {2023},
  doi = {10.1371/journal.pcbi.1010799},
}
Figure