Abstract
The recent emergence of accurate artificial intelligence (AI) models for disease diagnosis raises the possibility that AI-based clinical decision support could substantially lower the workload of healthcare providers. However, for this to occur, the input data to an AI predictive model, i.e., the patient’s features, must themselves be low-cost, that is, efficient, inexpensive, or low-effort to acquire. When time or financial resources for gathering data are limited, as in emergency or critical care medicine, modern high-accuracy AI models that use thousands of patient features are likely impractical. To address this problem, we developed the CoAI (Cost-aware AI) framework to enable any kind of AI predictive model to make accurate predictions given a small number of low-cost features. We show that CoAI dramatically reduces the cost of predicting prehospital acute traumatic coagulopathy, intensive care mortality, and outpatient mortality relative to existing risk scores, while improving prediction accuracy. It also outperforms existing state-of-the-art cost-sensitive prediction approaches in terms of predictive performance, model cost, and training time. Extrapolating these results to all trauma patients in the United States shows that, at a fixed false positive rate, CoAI could alert providers of tens of thousands more dangerous events than other risk scores while reducing providers’ data-gathering time by about 90 percent, leading to a savings of 200,000 cumulative hours per year across all providers. These benefits stem from several unique strengths: CoAI uses axiomatic feature attribution methods for precise importance estimation, is model-agnostic, and finds high-performance models within a given budget without tuning cost-vs-performance tradeoffs.
Introduction
Modern AI models are often "data-hungry," requiring hundreds or thousands of patient features to achieve high accuracy. In high-stakes environments like ambulances or Intensive Care Units (ICUs), healthcare providers lack the time to gather exhaustive data. Existing sparse models reduce feature counts but rarely account for heterogeneous costs—where a patient's age is "free" but a blood gas analysis takes 15 minutes. CoAI addresses this gap by providing a flexible framework to build models that respect real-world constraints without sacrificing predictive power.
The CoAI Framework
The CoAI framework follows a streamlined four-step process to optimize clinical predictions under constraints:
- Calculate Importance: A base model is trained on all features, and Shapley values (SHAP) are used to assign a precise, additive importance score ($\phi$) to each feature.
- Assign Costs: Each feature is annotated with a cost ($c$), such as time in minutes or financial cost in dollars.
- Solve the Knapsack Problem: For any given budget ($k$), a knapsack solver selects the subset of features that maximizes total importance while keeping total cost $\leq k$.
- Retrain: A final, specialized model is trained using only the selected features.
Unlike previous cost-sensitive methods, CoAI treats the budget as a hard constraint, making it more predictable for clinicians and significantly faster to train.
Experimental Evaluation
CoAI was evaluated across three distinct clinical domains:
- Trauma Care: Predicting Acute Traumatic Coagulopathy (ATC) using time-costs surveyed from EMS providers.
- Intensive Care: Predicting in-hospital mortality using feature count as a proxy for clinical complexity.
- Outpatient Care: Predicting 10-year mortality using financial costs based on Medicare fee-for-service rates.
Key Results and Clinical Impact
- Efficiency in Trauma: EMS providers typically have only 50 seconds for data entry. CoAI outperformed the standard PACT score (which requires 8 minutes of data) while staying within that 50-second budget. Nationally, this could save 200,000 provider hours per year.
- Critical Care: A CoAI model using just 3 features outperformed the 27-feature APACHE IVa score, a standard but cumbersome tool in critical care.
- Training Complexity: While competitor models often require over 100 training rounds to find a cost-accuracy tradeoff, CoAI achieved superior results with just two training rounds.
Discussion and Future Work
CoAI currently assumes additive costs, though in practice, multiple data points might be gathered simultaneously. Future work could explore "active" versions of CoAI that recommend the next best feature to gather based on a specific patient's initial results. Additionally, while Shapley values are powerful, the current optimization assumes feature independence; accounting for feature interactions could further refine performance.
Conclusion
CoAI provides a model-agnostic, efficient solution for deploying high-performance AI in resource-limited medical environments. By respecting real-world constraints, it offers a path toward dramatically improving patient care while reducing provider workload.
Summary updated