Abstract
Machine learning models in high-stakes healthcare environments often suffer from a lack of interpretability, leading to clinician skepticism. This paper introduces a framework that constructs a "medical narrative" by aggregating feature attributions and identifying influential "anchor" patients from training data. By mimicking the way physicians build intuition through experience, the proposed approach provides explanations that are not only more robust but also align closely with clinical reasoning. Experimental results on the MIMIC-III dataset for sepsis diagnosis demonstrate that aggregated methods, particularly Markovian SHAP ($M_{SHAP}$), significantly outperform standard attribution techniques in identifying critical clinical features.
Introduction
In intensive care units (ICUs), "black-box" models can accurately predict critical events like sepsis onset, yet their utility is limited if clinicians cannot understand the underlying reasoning. Current explainability tools, such as SHAP or Integrated Gradients, typically offer snapshots of feature importance for individual patients. However, they fail to capture the broader clinical narrative—the cumulative intuition a doctor develops over years of practice. This research aims to bridge this gap by identifying specific training examples that act as "anchors" for new predictions and analyzing how feature importance evolves during the model's learning trajectory.
Methodology
The authors utilize the MIMIC-III dataset, focusing on the sepsis diagnosis task with 17 key clinical features (e.g., Glasgow Coma Scale, blood pressure, and lab results). To build the medical narrative, they combine several techniques:
- Feature Attribution: Standard methods like SHAP and Integrated Gradients (IG) are used to identify driving clinical signs.
- Attribution Aggregation (AVA): The framework aggregates these attributions using Weighted ($W_{SHAP}$), Markovian ($M_{SHAP}$), and Borda Count ($B_{SHAP}$) methods to improve stability.
- Influence Functions: These are used to identify "anchor" patients in the training set that most heavily influenced a specific test prediction.
- Counterfactual Auditing: A method to assess how changes in influential past cases would alter the model's current feature space understanding.
Clinical Alignment and Narrative
A key goal of the framework is ensuring that model explanations align with medical reality. The authors demonstrate that the model identifies "anchor" patients that serve as precedents, much like a clinician's memory of past cases.
As shown in the feature ranking distributions, the model correctly prioritizes clinically relevant indicators such as the Sequential Organ Failure Assessment (SOFA) score and CDC sepsis criteria (appearing at the high-importance end of the rank spectrum). Furthermore, a "sanity check" on demographic features like race confirms that the model assigns them low importance (high rank values), indicating it is not relying on spurious or biased correlations for its predictions.
Experimental Results
The study evaluates the proposed aggregation methods against a "gold set" of ground-truth features derived from clinical decision trees.
The results indicate that aggregated methods consistently provide superior recall compared to raw SHAP or IG. Specifically, Markovian SHAP ($M_{SHAP}$) achieved a recall of 76% for certain architectures, a significant improvement over the 61% achieved by standard SHAP. Additionally, stability tests revealed that $M_{SHAP}$ and $W_{SHAP}$ are the most reliable methods, with $M_{SHAP}$ appearing in the top two performance rankings 99.5% of the time across 1,000 sampling iterations.
Discussion and Future Work
While the "medical narrative" approach shows promise in aligning ML explanations with clinical intuition, several areas for future research remain. The current framework treats the underlying model as a black box; incorporating internal architectural details might yield even deeper insights. Furthermore, while the results align with clinical knowledge, formal user studies with medical professionals are necessary to determine if these narratives actually improve diagnostic accuracy or trust in real-world ICU settings. Finally, the computational overhead of calculating influence functions and Shapley values must be optimized for real-time clinical applications.
Summary updated