Introduction
During public health crises, decision-makers often rely on static, "one-size-fits-all" measures, such as blanket school closures or fixed capacity limits. While these measures are straightforward to implement, they fail to account for the fluid nature of viral transmission and fluctuating levels of community risk. This research proposes a dynamic alternative: a reinforcement learning (RL) framework designed to optimize epidemic response policies in real-time. By focusing on the education sector as a proof of concept, the work demonstrates how RL can balance the competing goals of maintaining essential human interactions while minimizing infection risks.
The Problem: The Limitations of Static Policies
Traditional epidemic response strategies are often reactive and blunt. These instruments lack the flexibility to adapt to the rapidly changing dynamics of a virus. The author identifies a significant gap between these one-size-fits-all measures and the actual, fluid nature of epidemic spread. The core challenge lies in "navigating uncertainty": how can administrators make weekly decisions about classroom occupancy that are sensitive to both current community risk and the specific mechanics of indoor transmission? This research seeks to move from static rules to adaptive policies that can be recalibrated as conditions evolve.
Methodology: RL in a Stochastic Environment
The author develops a simulation environment where RL algorithms interact with stochastic epidemic models. The framework is applied to a classroom scenario involving $N$ students over a period of $W$ weeks.
The RL Framework
- State: Defined as a tuple $(c_w, E[I_w])$, representing the current community risk (off-campus infection probability) and the expected number of infected students in a given week.
- Action: The number of students permitted to attend class ($A_w$).
- State Evolution: Driven by stochastic models, such as an approximate SIR (Susceptible-Infectious-Recovered) model or specific indoor classroom models that account for viral dose, transmission probability, and ventilation.
- Reward Function: A weighted difference between the number of allowed students and the expected number of infections. This weighting serves as a tuning knob, allowing administrators to prioritize either educational continuity or infection suppression.
Simulation-Based Data
Rather than relying on static historical datasets, the framework utilizes a simulation-based approach. The environment generates data using stochastic epidemic models adapted from existing literature (Ondula & Krishnamachari, 2022; Hekmati et al., 2022) to simulate disease spread behaviors and probabilistic models of off-campus infection.
Key Insights: Pareto Optimality and Dynamic Adaptation
A central finding of this work is the conceptualization of epidemic management as a multi-objective optimization problem. By adjusting the weights in the reward function, the RL agent can identify Pareto-optimal strategies—scenarios where attendance cannot be increased without also increasing infection risk.
Unlike a static rule (e.g., "always 50% capacity"), the RL agent learns to adapt to the environment. It might allow 100% capacity when community risk is low but sharply pivot to 0% as soon as a specific risk threshold is crossed. This dynamic approach can potentially result in higher total attendance over a semester than a conservative, static rule would allow.
Preliminary Results and Future Directions
As this research represents ongoing doctoral work, the results are currently in the proof-of-concept stage:
- Feasibility: The author has successfully integrated infection models with RL modules into a functional software platform.
- Policy Refinement: Preliminary trials using Q-learning show that the agent effectively learns to refine occupancy prescriptions based on infection counts and community patterns.
- Trade-off Mapping: The research has successfully identified different Pareto-optimal trade-offs, providing a "menu" of policies ranging from highly cautious to more education-focused.
Limitations and Timeline
The effectiveness of the RL policy remains highly dependent on the accuracy of the underlying epidemic model parameters, which are often noisy in real-world settings. Future work will focus on addressing parameter sensitivity, refining the reward functions, and extending the framework to healthcare and economic sectors. A full thesis proposal is expected in 2025, with a final defense projected for 2026.
Summary updated