CoAI: Cost-Aware Artificial Intelligence for Health Care

Gabriel Erion, Joseph D. Janizek, Carly Hudelson, Richard B. Utarnachitt, Andrew M. McCoy, Michael R. Sayre, Nathan J. White, Su‐In LeemedRxiv 2021 · DOI

Abstract

The recent emergence of accurate artificial intelligence (AI) models for disease diagnosis raises the possibility that AI-based clinical decision support could substantially lower the workload of healthcare providers. However, for this to occur, the input data to an AI predictive model, i.e., the patient’s features, must themselves be low-cost, that is, efficient, inexpensive, or low-effort to acquire. When time or financial resources for gathering data are limited, as in emergency or critical care medicine, modern high-accuracy AI models that use thousands of patient features are likely impractical. To address this problem, we developed the CoAI (Cost-aware AI) framework to enable any kind of AI predictive model to make accurate predictions given a small number of low-cost features. We show that CoAI dramatically reduces the cost of predicting prehospital acute traumatic coagulopathy, intensive care mortality, and outpatient mortality relative to existing risk scores, while improving prediction accuracy. It also outperforms existing state-of-the-art cost-sensitive prediction approaches in terms of predictive performance, model cost, and training time. Extrapolating these results to all trauma patients in the United States shows that, at a fixed false positive rate, CoAI could alert providers of tens of thousands more dangerous events than other risk scores while reducing providers’ data-gathering time by about 90 percent, leading to a savings of 200,000 cumulative hours per year across all providers. These benefits stem from several unique strengths: CoAI uses axiomatic feature attribution methods for precise importance estimation, is model-agnostic, and finds high-performance models within a given budget without tuning cost-vs-performance tradeoffs.

Introduction

Modern AI models are often "data-hungry," requiring hundreds or thousands of patient features to achieve high accuracy. In high-stakes environments like ambulances or Intensive Care Units (ICUs), healthcare providers lack the time to gather exhaustive data. Existing sparse models reduce feature counts but rarely account for heterogeneous costs—where a patient's age is "free" but a blood gas analysis takes 15 minutes. CoAI addresses this gap by providing a flexible framework to build models that respect real-world constraints without sacrificing predictive power.

The CoAI Framework

The CoAI framework follows a streamlined four-step process to optimize clinical predictions under constraints:

  1. Calculate Importance: A base model is trained on all features, and Shapley values (SHAP) are used to assign a precise, additive importance score ($\phi$) to each feature.
  2. Assign Costs: Each feature is annotated with a cost ($c$), such as time in minutes or financial cost in dollars.
  3. Solve the Knapsack Problem: For any given budget ($k$), a knapsack solver selects the subset of features that maximizes total importance while keeping total cost $\leq k$.
  4. Retrain: A final, specialized model is trained using only the selected features.
The CoAI framework overview, showing (a) feature importance calculation, (b) cost annotation, and (c) the cost-aware feature selection and retraining process.

Unlike previous cost-sensitive methods, CoAI treats the budget as a hard constraint, making it more predictable for clinicians and significantly faster to train.

Experimental Evaluation

CoAI was evaluated across three distinct clinical domains:

  • Trauma Care: Predicting Acute Traumatic Coagulopathy (ATC) using time-costs surveyed from EMS providers.
  • Intensive Care: Predicting in-hospital mortality using feature count as a proxy for clinical complexity.
  • Outpatient Care: Predicting 10-year mortality using financial costs based on Medicare fee-for-service rates.

Key Results and Clinical Impact

  • Efficiency in Trauma: EMS providers typically have only 50 seconds for data entry. CoAI outperformed the standard PACT score (which requires 8 minutes of data) while staying within that 50-second budget. Nationally, this could save 200,000 provider hours per year.
  • Critical Care: A CoAI model using just 3 features outperformed the 27-feature APACHE IVa score, a standard but cumbersome tool in critical care.
  • Training Complexity: While competitor models often require over 100 training rounds to find a cost-accuracy tradeoff, CoAI achieved superior results with just two training rounds.

Discussion and Future Work

CoAI currently assumes additive costs, though in practice, multiple data points might be gathered simultaneously. Future work could explore "active" versions of CoAI that recommend the next best feature to gather based on a specific patient's initial results. Additionally, while Shapley values are powerful, the current optimization assumes feature independence; accounting for feature interactions could further refine performance.

Conclusion

CoAI provides a model-agnostic, efficient solution for deploying high-performance AI in resource-limited medical environments. By respecting real-world constraints, it offers a path toward dramatically improving patient care while reducing provider workload.

Summary updated

Abstract

Abstract The recent emergence of accurate artificial intelligence (AI) models for disease diagnosis raises the possibility that AI-based clinical decision support could substantially lower the workload of healthcare providers. However, for this to occur, the input data to an AI predictive model, i.e., the patient’s features, must themselves be low-cost, that is, efficient, inexpensive, or low-effort to acquire. When time or financial resources for gathering data are limited, as in emergency or critical care medicine, modern high-accuracy AI models that use thousands of patient features are likely impractical. To address this problem, we developed the CoAI ( Co st-aware AI ) framework to enable any kind of AI predictive model (e.g., deep neural networks, tree ensemble models, etc.) to make accurate predictions given a small number of low-cost features. We show that CoAI dramatically reduces the cost of predicting prehospital acute traumatic coagulopathy, intensive care mortality, and outpatient mortality relative to existing risk scores, while improving prediction accuracy. It also outperforms existing state-of-the-art cost-sensitive prediction approaches in terms of predictive performance, model cost, and training time. Extrapolating these results to all trauma patients in the United States shows that, at a fixed false positive rate, CoAI could alert providers of tens of thousands more dangerous events than other risk scores while reducing providers’ data-gathering time by about 90 percent, leading to a savings of 200,000 cumulative hours per year across all providers. We extrapolate similar increases in clinical utility for CoAI in intensive care. These benefits stem from several unique strengths: First, CoAI uses axiomatic feature attribution methods that enable precise estimation of feature importance. Second, CoAI is model-agnostic, allowing users to choose the predictive model that performs the best for the prediction task and data at hand. Finally, unlike many existing methods, CoAI finds high-performance models within a given budget without any tuning of the cost-vs-performance tradeoff. We believe CoAI will dramatically improve patient care in the domains of medicine in which predictions need to be made with limited time and resources.
BibTeX
@misc{erion2021coai,
  author = {Gabriel Erion and Joseph D. Janizek and Carly Hudelson and Richard B. Utarnachitt and Andrew M. McCoy and Michael R. Sayre and Nathan J. White and Su‐In Lee},
  title = {{CoAI: Cost-Aware Artificial Intelligence for Health Care}},
  year = {2021},
  doi = {10.1101/2021.01.19.21249356},
  howpublished = {medRxiv},
}
Figure