Abstract
Accurately assigning cause of death is vital to understanding health outcomes and improving healthcare provision. While cancer-specific cause of death is a key outcome in clinical trials, assignment from death certification is prone to misattribution. We developed an interpretable machine learning classifier to predict prostate cancer death from free-text summaries of medical history for prostate cancer patients (CAP). Compared to independent human expert assignment, the classifier showed >90% accuracy. Visualizations were developed to highlight predictive elements, improving analyst confidence and revealing potential for decision-support tools.
Introduction
Determining the exact cause of death (CoD) is more complex than it appears. Death certificates, while official, are prone to misattribution—particularly in oncology, where patients may die with a disease rather than from it. In clinical trials, this noise can skew mortality outcome measures, often requiring expensive and time-consuming manual reviews by expert committees.
Researchers from the University of Bristol and the University of Exeter have developed a machine learning approach to automate this process using free-text medical summaries. A primary focus of this work is interpretability, ensuring that the "black box" of AI is transparent and useful for clinical decision-making.
Methods
The research team constructed a classification pipeline using a "bag-of-words" representation of health records. The process involved:
- Preprocessing: Text was lemmatized using the WordNetLemmatizer to reduce words to their base forms (e.g., "testing" to "test").
- Classification: Three standard algorithms were compared: Support Vector Classifiers (SVC), Logistic Regression (LR), and Random Forest (RF).
- Interpretability: Four methods—Gini Importance, TreeInterpreter, LIME, and SHAP—were used to quantify word-level contributions to predictions.
- Visualization: "Interpretable vignettes" were created, where font size and color in the original summaries indicate a word's influence on the prediction (e.g., blue for "prostate cancer death").
Dataset
The study utilized the CAP (Cluster randomised trial of PSA testing for prostate cancer) dataset, which includes 2,606 medical summaries of men diagnosed with prostate cancer who died by March 2016. Ground truth labels were established by an independent committee of urologists, pathologists, and oncologists.
Results
All three classifiers demonstrated high performance. The Support Vector Classifier (SVC) achieved the highest Area Under the Curve (AUC) at 0.9749, while the Random Forest (RF) model provided a balanced performance with an accuracy of 91.8% and an F1 score of 0.931.
Key findings include:
- Clinical Alignment: The model prioritized features like "bone scan," "spine," and "hormone," which are clinically recognized signs of progressing prostate cancer.
- Contextual Nuance: The "interpretable vignettes" revealed that the absence of certain keywords could be as predictive as their presence.
- Expert Trust: Preliminary feedback indicated that visualizing the model's reasoning significantly improved clinician confidence in the tool's potential as a decision-support aid.
Conclusion
The results suggest that an interpretability-first approach can produce high-accuracy tools that align with clinical expertise. While the current model struggles with negation and lacks numerical data (like PSA values), it represents a significant step toward automated, transparent cause-of-death assignment. The researchers have published their code on GitHub to facilitate further application in clinical settings.
Summary updated