Resource Planning for Hospitals Under Special Consideration of the COVID-19 Pandemic: Optimization and Sensitivity Analysis

Thomas Bartz-Beielstein, Marcel Dröscher, Alpar Gür, Alexander Hinterleitner, Olaf Mersmann, Dessislava Peeva, Lennard Reese, Nicolas Rehbach, Frederik Rehbach, Amrita Sen, Aleksandr Subbotin, Martin ZaeffererGECCO Companion 2021 · OpenReview

TL;DR

This paper tackles the complex challenge of hospital resource planning during the COVID-19 pandemic by developing a simulation-based optimization approach. They introduce a novel Surrogate Model-Based Sensitivity Analysis (SMBSA) to efficiently identify critical parameters for their hospital simulation model, significantly reducing computational burden and improving prediction accuracy.

What Problem Does This Paper Address?

The COVID-19 pandemic placed immense strain on healthcare systems, necessitating robust planning for critical resources like hospital beds and ventilators. Accurately predicting demand and optimizing resource allocation is crucial for effective decision-making by healthcare institutions and policymakers. However, existing simulation models for this purpose are often computationally expensive and highly sensitive to numerous parameters, making optimization a significant challenge. This paper addresses the need for an efficient and accurate method to optimize these complex simulation models, particularly in the context of dynamic and high-dimensional real-world problems like pandemic response.

Methods

The authors employ a multi-faceted approach combining discrete-event simulation (DES) with optimization and sensitivity analysis.

  • Discrete-Event Simulation (DES): They utilize the simmer R-package to build BABSIM.HOSPITAL, a DES model that simulates patient trajectories through a hospital. This model is parameterized by 29 variables representing transition probabilities, durations, and distribution properties.
  • Surrogate Model-Based Optimization (SMBO): Recognizing the computational cost of direct simulation-based optimization, they adopt an SMBO approach using the SPOT R-package. SMBO fits a cheaper-to-evaluate surrogate model to the expensive simulation objective function, allowing for faster exploration of the parameter space.
  • Surrogate Model-Based Sensitivity Analysis (SMBSA): A key contribution is the development of SMBSA. This method leverages the surrogate models created during the SMBO process to perform sensitivity analysis. By analyzing these surrogate models (Kriging, Linear Regression, and Random Forest), they can identify parameters that have the most significant impact on the simulation's outcome. This allows for a reduction in the problem's dimensionality by focusing optimization efforts on the most influential parameters.

Datasets

The paper uses real-world data for hospital resource planning in Germany. Specifically, they integrate data from:

  • Robert Koch Institute (RKI): This dataset contains over 780,000 observations with 18 variables related to COVID-19 cases (e.g., age, gender, date of infection).
  • Deutsche interdisziplinäre Vereinigung für Intensiv- und Notfallmedizin (DIVI): This data provides information on Intensive Care Unit (ICU) capacity in Germany.

The data is automatically collected and curated using an Extract, Transform, Load (ETL) process.

Key Insights and Ideas

  • "Out-of-the-box" optimizers fail: Standard optimization algorithms struggled to find improved parameter configurations within the tight time constraints imposed by the noisy and high-dimensional nature of the problem.
  • SMBO is effective but computationally intensive: While SMBO provided good results, it still increased the computational burden.
  • Sensitivity analysis is crucial for efficiency: SMBSA proved vital for understanding the "fitness landscape" of the optimization problem, enabling the identification of parameters that truly matter.
  • Parameter importance is not always intuitive: Domain analysis (expert knowledge) provided initial insights, but data-driven analysis revealed nuances, with some parameters identified as important by experts showing less impact in practice, and vice-versa.
  • Reducing dimensionality is key: Focusing optimization on a subset of the most important parameters can drastically decrease optimizer runtime without significant loss in quality.

Key Results

  • The SMBSA approach successfully identified a core set of parameters (specifically x13, x14, and x26) that significantly influence the simulation's accuracy.
  • By removing less important parameters from the optimization loop, the authors demonstrated a potential for drastically decreasing optimizer runtime.
  • The paper successfully automated data collection and curation, enabling continuous simulation and optimization runs.
  • The simmer DES software proved capable of handling large datasets (over half a million infections) within reasonable time constraints.

Limitations and Future Work

  • Noise in the simulator: The inherent noise in the simulation output made it challenging to observe clear changes in optimizer efficiency when adding or removing parameters, requiring further investigation.
  • Parameter importance variability: While some parameters consistently showed high importance, identifying the least important parameters was less straightforward, with some parameters showing unexpected impacts.
  • Future Work: The authors plan to conduct further benchmark studies on alternative SMBO approaches and optimizers. They also highlight the potential of their SMBSA approach for other complex, high-dimensional modeling and simulation problems beyond hospital resource planning, such as elevator systems and academic study period simulations. The BABSIM.HOSPITAL simulator is made available as an open-source benchmark.

Summary updated

Abstract

Crises like the COVID-19 pandemic pose a serious challenge to health-care institutions. They need to plan the resources required for handling the increased load, for instance, hospital beds and ventilators. To support the resource planning of local health authorities from the Cologne region, BaBSim.Hospital, a tool for capacity planning based on discrete event simulation, was created. The predictive quality of the simulation is determined by 29 parameters. Reasonable default values of these parameters were obtained in detailed discussions with medical professionals. We aim to investigate and optimize these parameters to improve BaBSim.Hospital. First approaches with "out-of-the-box" optimization algorithms failed. Implementing a surrogate-based optimization approach generated useful results in a reasonable time. To understand the behavior of the algorithm and to get valuable insights into the fitness landscape, an in-depth sensitivity analysis was performed. The sensitivity analysis is crucial for the optimization process because it allows focusing the optimization on the most important parameters. We illustrate how this reduces the problem dimension without compromising the resulting accuracy. The presented approach is applicable to many other real-world problems, e.g., the development of new elevator systems to cover the last mile or simulation of student flow in academic study periods.
BibTeX
@inproceedings{bartzbeielstein2021resource,
  author = {Thomas Bartz-Beielstein and Marcel Dröscher and Alpar Gür and Alexander Hinterleitner and Olaf Mersmann and Dessislava Peeva and Lennard Reese and Nicolas Rehbach and Frederik Rehbach and Amrita Sen and Aleksandr Subbotin and Martin Zaefferer},
  title = {{Resource Planning for Hospitals Under Special Consideration of the COVID-19 Pandemic: Optimization and Sensitivity Analysis}},
  booktitle = {GECCO Companion},
  year = {2021},
  url = {https://openreview.net/forum?id=g4BIxp8luf},
}
Figure