TL;DR
This paper introduces a novel Generalized Additive Model with Neural Networks (GAM-NN) architecture designed to predict postoperative in-hospital mortality. By combining the predictive power of neural networks with the interpretability of additive models, GAM-NNs offer clinicians a transparent view into the factors driving mortality risk, fostering trust and enabling more targeted interventions.
What Problem Does This Paper Address?
Deep neural networks (DNNs) have demonstrated impressive accuracy in predicting complex medical outcomes like postoperative in-hospital mortality. However, their "black box" nature poses a significant barrier to clinical adoption. Clinicians need not only accurate predictions but also an understanding of why a prediction is made to trust the model, tailor treatments, and gain insights into disease mechanisms. Traditional models like logistic regression (LR) are interpretable but often lack the capacity to capture complex, non-linear relationships in the data, which are common in biological and medical systems. This paper addresses the critical need for models that are both highly accurate and clinically interpretable, bridging the gap between cutting-edge machine learning and practical healthcare applications.
Methods
The core innovation is the development of a Generalized Additive Model with Neural Networks (GAM-NN). This architecture builds upon the concept of Generalized Additive Models (GAMs), where the overall prediction is a sum of individual (or pairwise) functions of input features. In the GAM-NN, each input feature is processed by its own small neural network (a "sub-network"). The outputs of these sub-networks are then linearly combined to produce the final prediction. This allows the model to learn complex, non-linear relationships for each feature independently, while the additive structure ensures that the contribution of each feature can be visualized and understood.
Key aspects of the methodology include:
- Feature Engineering: The study augmented existing feature sets with new ones, including total anesthesia case time, time spent with low mean arterial pressure (MAP) during surgery, and Healthcare Cost and Utilization Project (HCUP) code descriptions.
- Model Architecture: Each input feature is fed into a separate neural network (with a single hidden layer and tanh activation), and their outputs are summed. Binary features have a direct connection to the final logistic layer.
- Interpretability Mechanism: The paper focuses on visualizing the learned "contributions" of each feature. For continuous features, this involves plotting the feature's value against its learned contribution to mortality risk, allowing direct comparison with linear contributions from LR models.
- Training and Validation: Data was split into 80% training and 20% testing sets. The model was trained using cross-entropy loss with class weights to handle the imbalanced dataset (low mortality rate).
Datasets
The study utilized a dataset of 59,985 surgical records from a single institution (UCLA Medical Center) collected between 2013 and 2016. This dataset was previously used in related work. The original feature set of 46 features was augmented with 40 new features, resulting in a total of 86 features for the GAM-NN model. The occurrence of in-hospital mortality was low, at 0.81% in the training set and 0.72% in the testing set. No new benchmarks were introduced; the focus was on validating the proposed GAM-NN against existing models like logistic regression.
Key Insights and Ideas
- Bridging the Accuracy-Interpretability Divide: The GAM-NN successfully demonstrates that it's possible to achieve high predictive accuracy comparable to complex DNNs while retaining the interpretability of simpler models like LR. This is a crucial step towards making advanced AI models trustworthy and actionable in clinical settings.
- Non-linear Contributions Matter: The visualization of feature contributions revealed that GAM-NNs can learn non-linear relationships that are often more clinically intuitive than the linear relationships captured by LR. For example, the risk associated with mean arterial pressure (MAP) might not be strictly linear, with both very low and very high MAP potentially increasing risk.
- HCUP Features Enhance Prediction: Including HCUP (Healthcare Cost and Utilization Project) procedure codes significantly improved the model's predictive performance, highlighting the value of structured clinical information beyond basic vital signs and demographics.
- Interpretability as a Tool for Insight: The ability to visualize feature contributions allows clinicians to not only understand individual predictions but also to identify patterns and potential areas for intervention. For instance, if low MAP is a strong predictor of mortality, clinicians can focus on managing blood pressure more aggressively.
Key Results
- High Predictive Accuracy: The GAM-NN model incorporating HCUP features achieved the highest Area Under the Curve (AUC) of 0.921 (95% CI: 0.895–0.95), outperforming logistic regression models.
- Comparable Performance: Both GAM-NN models (with and without HCUP features) showed higher AUCs than their LR counterparts.
- Trade-off in Precision: While AUC was higher, GAM-NN models had slightly lower average precisions compared to LR models. The LR model without HCUP features achieved the highest average precision of 0.217 (95% CI: 0.136–0.31).
- Visualized Non-linearity: Figure 2 clearly illustrates how GAM-NN contributions can be non-linear (e.g., for MAX_DES, AVG_DBP, AVG_MAP_10_min), contrasting with the linear contributions of LR.
- Patient-Specific Insights: Table 4 shows how different features contribute to mortality risk for individual patients, demonstrating the model's potential for personalized risk assessment.
Limitations and Future Work
- Single Institution Data: The study acknowledges that the model was developed and validated on data from a single institution (UCLA Medical Center) and from a specific time period (2013-2016). This limits the generalizability of the findings, as clinical practices and data recording can vary significantly across institutions and over time.
- Retrospective Data: The reliance on retrospective electronic health record data means that the model's performance might differ in real-time clinical settings.
- Correlation vs. Causation: The authors emphasize that feature contributions represent correlations, not necessarily causation. Modifying a feature (e.g., not placing an arterial line) based on its contribution might not directly prevent mortality.
- Future Work: The authors plan to address these limitations by:
- Collaborating with other institutions to validate the model on diverse datasets.
- Accessing more recent data within their own institution to assess temporal generalizability.
- Investigating how clinicians interact with and utilize the interpretability features of the model in practice.
- Exploring the potential for using the GAM-NN as a clinical decision support tool, providing actionable insights to clinicians.
Summary updated