TL;DR
This paper introduces a novel method using knowledge distillation to create faithful and plausible explanations for complex medical code prediction models. By training a simpler "student" model to mimic a powerful "teacher" model, they achieve interpretable outputs that accurately reflect the teacher's reasoning, addressing a critical need for trust and understanding in clinical AI.
What Problem Does This Paper Address?
Machine learning models have shown great promise in healthcare, particularly in tasks like predicting medical codes from clinical notes. However, these high-performing models are often "black boxes," making it difficult for clinicians to understand why a prediction was made. This lack of transparency hinders adoption, as healthcare professionals need to trust and verify model decisions, especially in high-stakes situations. Existing explainability methods often struggle to balance faithfulness (accurately representing the model's true reasoning) with plausibility (making sense to a human expert), and can be computationally expensive or provide only local explanations. This paper aims to bridge this gap by generating faithful and plausible explanations for complex medical code prediction models.
Methods
The core of this paper's approach is knowledge distillation. Instead of training a model directly on the medical data to predict ICD codes, they train a simpler "student" model to mimic the predictions of a pre-trained, complex "teacher" model. This means the student model learns to replicate the teacher's behavior, not necessarily the ground truth labels directly.
Specifically, they use a linear bag-of-words regression model as the student. This choice is deliberate: linear models are inherently interpretable, decomposable, and simulatible, aligning with criteria for good explanations. The student model is trained to minimize the squared difference between its predicted probabilities and the teacher model's output probabilities for each ICD code. This process is performed independently for each ICD code.
For generating natural language explanations from the student model, they extract feature importance weights and average them over n-grams in the clinical text, selecting the n-gram with the highest average weight.
Datasets
The primary dataset used is MIMIC-III, a publicly available dataset containing anonymized ICU patient records, including clinical notes and ICD-9 codes. The authors focus on discharge summaries. They utilize three pre-trained "teacher" models for evaluation:
- DR-CAML: A model designed for explainable predictions.
- Hierarchical Attention Networks (HAN): A Bi-GRU document classifier.
- TransICD: A transformer-based method.
They also use a logistic regression baseline trained directly on the ICD-9 codes for comparison.
Key Insights and Ideas
- Disentangling Faithfulness and Plausibility: The paper argues that knowledge distillation offers a way to achieve both faithfulness (by mimicking the teacher) and plausibility (by using an interpretable student model).
- Student Model as a Proxy: The student model acts as a faithful proxy for the teacher, allowing for explanations without altering the original complex model.
- Global Explanations: Unlike many post-hoc methods that provide prediction-specific explanations, the student model's learned weights are global, offering a consistent view of its reasoning.
- Simplicity for Interpretability: The choice of a linear bag-of-words model for the student is crucial for its inherent interpretability, making it easier for domain experts to understand.
Key Results
- Faithfulness: The student models demonstrated high faithfulness to their respective teacher models, significantly outperforming a logistic regression baseline in terms of correlation and classification metrics (AUC, F1) when comparing student predictions to teacher predictions. For example, Student-DRCAML achieved a Spearman correlation of 0.794 compared to the logistic baseline's 0.036.
- Plausibility: While quantitative plausibility evaluation is challenging, qualitative comparisons and a classifier-based evaluation suggest that the student model's explanations are comparably plausible to those of the original teacher models, and in some cases, more so. Table 3 shows examples where student explanations are nearly identical to DR-CAML's.
- Efficiency: The student models are significantly smaller and faster to train (using only 50k parameters) than the teacher models, making them practical for generating explanations.
Limitations and Future Work
- Dataset Specificity: The current work is limited to a single English-language dataset (MIMIC-III) and the task of medical code prediction. Future work should explore applicability to new domains and tasks.
- Linear Student Limitations: There might be black-box models for which no linear student model can be sufficiently faithful.
- Explanation Format: The current method primarily generates n-gram based explanations. Future work could explore other explanation types, such as counterfactual explanations.
- Plausibility Evaluation: The plausibility evaluation relied on a limited set of annotations. Future work should involve more extensive human evaluations with metrics like sufficiency and simulatibility.
- Fine-tuning Student: The authors suggest exploring fine-tuning the student model to better predict ground-truth ICD codes, which could improve its predictive performance while retaining interpretability.
Summary updated