Predicting Pediatric Surgical Durations

Neal Master, David Scheinker, Nicholas BambosarXiv 2016 · DOI · arXiv

Abstract Effective management of operating room resources relies on accurate predictions of surgical case durations. This prediction problem is known to be particularly difficult in pediatric hospitals due to the extreme variation in pediatric patient populations. We propose a novel metric for measuring accuracy of predictions which captures key issues relevant to hospital operations. With this metric in mind we propose several tree-based prediction models. Some are automated (they do not require input from surgeons) while others are semi-automated (they do require input from surgeons). We see that many of our automated methods generally outperform currently used algorithms and even achieve the same performance as surgeons. Our semi-automated methods can outperform surgeons by a significant margin. We gain insights into the predictive value of different features and suggest avenues of future work.

Introduction

Operating rooms (ORs) are the financial and operational engine of a hospital, with costs estimated at approximately $4,000 per hour. Inefficient scheduling leads to cancelled surgeries, lost revenue, and increased stress for both staff and patients. Predicting surgical durations is critical for optimization, yet it remains notoriously difficult in pediatric settings. Unlike adult populations, pediatric patients exhibit extreme variation in age, size, and developmental stages; a procedure performed on a neonate differs fundamentally from the same procedure on an adolescent.

Currently, hospitals typically rely on historical averages or subjective surgeon estimates. Historical averages often ignore patient-specific nuances, while surgeon "guesstimates" can be biased by individual experience or institutional incentives. Previous research has often been pessimistic about the utility of machine learning in this domain, suggesting that standard clinical variables do not correlate well with pediatric case times. This study challenges that view by introducing novel metrics and robust tree-based models.

Methodology

Operational Metric and Log-Space Training

Standard error metrics like Root Mean Square Error (RMSE) often fail to reflect the operational realities of a hospital. For instance, a 15-minute error is negligible for a 4-hour surgery but catastrophic for a 20-minute one. To address this, we proposed a "correctness" metric, $\tau(\hat{Y})$, which considers a prediction accurate if it falls within a specific percentage of the actual duration, subject to minimum and maximum allowable windows.

Furthermore, surgical durations are typically "heavy-tailed," with a majority of short cases and a few extremely long ones. To handle this skewness, we trained our models on the natural logarithm of the duration. This transformation makes the data distribution more symmetric, aligning better with our operational metric and improving model robustness.

Distribution of surgical durations in log-space, showing improved symmetry compared to raw minutes.

Model Selection

We evaluated three primary tree-based ensemble models:

  • Decision Tree Regressor (DTR): A single tree model.
  • Random Forest Regressor (RFR): An ensemble of trees that reduces variance.
  • AdaBoost Regressor (ABR): An adaptive boosting method that focuses on difficult-to-predict cases.

These were categorized into automated models (using EHR data such as age, weight, and procedure name) and semi-automated models (which incorporate the surgeon’s estimate as an additional feature).

Dataset

The study utilized data from the Lucile Packard Children’s Hospital (LPCH) Stanford, spanning May 2014 to January 2016. To ensure statistical reliability, we filtered the initial 4,475 unique procedure names down to the 12 most frequent types (e.g., Adenoidectomy, Appendectomy, Bronchoscopy), resulting in 917 observations for training and cross-validation.

Results

Our analysis revealed a clear performance hierarchy: Historical Averages $\prec$ DTR $\prec$ Surgeon Estimates $\prec$ RFR $\approx$ ABR $\prec$ Semi-Automated Ensembles.

The semi-automated Random Forest (RFR-SCH) achieved the highest overall accuracy at 71%, significantly outperforming the surgeons' baseline of 62% and the historical average of 44%. In specific cases, such as bronchoscopies, ensemble models outperformed human estimates by a factor of three.

Feature Importance

The models provided valuable insights into which variables drive surgical duration. For automated models, the procedure name was the most dominant feature, followed by primary surgeon identity and patient weight.

Relative importance of features across different prediction models.

Contrary to some prior literature, we found that patient weight serves as a powerful proxy for developmental complexity in pediatrics. Additionally, the identity of the primary surgeon was a significant predictor, likely reflecting variations in individual technique and speed.

Discussion and Future Work

The success of automated ensemble models (RFR and ABR) in matching or exceeding surgeon performance suggests that automation can significantly reduce the administrative burden on medical staff. The "hybrid" approach—using machine learning to correct human bias—yielded the most accurate results.

However, several challenges remain. "Concept drift" is a concern; if surgeons begin to rely on or react to AI predictions, their own estimates might change, potentially affecting model performance. Additionally, extending these models to the "long-tail" of rare pediatric surgeries requires further research, possibly involving Bayesian methodologies to better integrate expert opinion with sparse data. Future iterations could also benefit from more granular clinical features, such as the presence of surgical residents or specific intraoperative complexities.

Summary updated

Abstract

Effective management of operating room resources relies on accurate predictions of surgical case durations. This prediction problem is known to be particularly difficult in pediatric hospitals due to the extreme variation in pediatric patient populations. We propose a novel metric for measuring accuracy of predictions which captures key issues relevant to hospital operations. With this metric in mind we propose several tree-based prediction models. Some are automated (they do not require input from surgeons) while others are semi-automated (they do require input from surgeons). We see that many of our automated methods generally outperform currently used algorithms and even achieve the same performance as surgeons. Our semi-automated methods can outperform surgeons by a significant margin. We gain insights into the predictive value of different features and suggest avenues of future work.
BibTeX
@misc{master2016predicting,
  author = {Neal Master and David Scheinker and Nicholas Bambos},
  title = {{Predicting Pediatric Surgical Durations}},
  year = {2016},
  eprint = {1605.04574},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1605.04574},
}
Figure