Explainable Artificial Intelligence Helps in Understanding the Effect of Fibronectin on Survival of Sepsis

Anna Lemańska-Perek, Dorota Krzyżanowska‐Gołąb, Katarzyna Kobylińska, Przemysław Biecek, Tomasz Skalec, Maciej Tyszko, Waldemar Goździk, Barbara AdamikCells 2022 · DOI

TL;DR

This paper leverages explainable AI (XAI) to build a robust model predicting sepsis survival, highlighting fibronectin (FN) as a key, previously under-understood, predictor. By combining traditional clinical scores with FN levels, they achieve high accuracy and offer a personalized risk calculator, paving the way for more tailored patient care.

What Problem Does This Paper Address?

Sepsis is a life-threatening condition with high mortality, making accurate survival prediction crucial for resource allocation and patient management. While clinical scores like APACHE II and SOFA are standard, identifying additional biomarkers that improve prediction accuracy remains an active area of research. Fibronectin (FN), a protein involved in host response to infection, has been observed to decrease in sepsis patients, but its precise impact on survival has not been clearly elucidated. Traditional "black box" machine learning models can predict outcomes well, but their lack of interpretability hinders clinical trust and adoption. This paper aims to bridge this gap by using explainable AI to understand how FN, alongside other clinical factors, contributes to sepsis survival prediction.

Methods

The researchers employed a machine learning approach, specifically focusing on the Random Forest algorithm, which demonstrated the best performance among the tested models (logistic regression, random forest, and gradient boosting). To address the "black box" nature of these models, they integrated Explainable Artificial Intelligence (XAI) techniques.

Key XAI methods used include:

  • Global-level methods:
    • Feature Importance: This method assesses how much a model's performance (measured by AUC loss) degrades when a specific feature's values are permuted. This helps identify the overall significance of each input variable.
  • Local-level methods: These techniques provide insights into individual patient predictions:
  • Break Down: Decomposes a model's prediction for a specific instance into contributions from each feature.
  • Shapley Additive Explanations (SHAP values): A game-theoretic approach to attribute the prediction to each feature, averaging contributions across all possible feature orderings.
  • Ceteris Paribus: Illustrates how a prediction would change if the value of a single feature were altered, keeping others constant.

These methods were implemented using the R programming language and associated libraries (mlr, DALEX).

Datasets

The study utilized a dataset of 127 consecutive patients admitted to the Intensive Care Unit (ICU) with sepsis or septic shock between January 2018 and December 2019. Of these, 122 patients met the inclusion criteria. Data collected included:

  • Plasma fibronectin (pFN) concentrations
  • Cellular fibronectin (EDA-FN) concentrations
  • Clinical scores: APACHE II, SOFA
  • Demographic data: Age
  • Laboratory parameters: INR, procalcitonin, C-reactive protein, platelet count, d-dimers, leukocytes

This dataset was used for training and evaluating the machine learning models. The authors also developed a web-based risk calculator, which is intended to be continuously updated with new data, suggesting a pathway for future dataset expansion.

Key Insights and Ideas

  • Fibronectin's nuanced role: While previous studies noted decreased FN in sepsis, this work quantifies its significant contribution to survival prediction, particularly the plasma form (pFN). This moves beyond simply observing a change to understanding its predictive power.
  • XAI for clinical trust: The application of XAI techniques is crucial. It not only validates the model's predictions but also provides physicians with interpretable insights into why a patient is predicted to survive or not, fostering trust and enabling more informed clinical decisions.
  • Personalized risk assessment: The development of a web-based risk calculator powered by XAI allows for individual patient risk exploration. This moves towards personalized medicine, where treatment strategies can be tailored based on a deeper understanding of individual risk factors.
  • Synergy of biomarkers and clinical scores: The model's high performance is achieved by combining FN levels with established clinical indicators like INR, age, and APACHE II, demonstrating the power of multi-modal data integration.

Key Results

  • Model Performance: The Random Forest model achieved a successful classification rate (AUC) of 0.92.
  • Sensitivity (recall): 0.92
    • Positive Predictive Value (precision): 0.76
    • Accuracy: 0.79
  • Key Predictors: The most important indicators for predicting sepsis survival were identified as:
  • INR (International Normalized Ratio)
    1. Plasma Fibronectin (pFN) concentration
    2. Age
    3. APACHE II score
  • Fibronectin Levels: Lower plasma FN concentrations were significantly associated with non-survival (p < 0.001). EDA-FN levels did not show a statistically significant difference between survivors and non-survivors.
  • Feature Importance: The INR and pFN were the most significant variables in the Random Forest model, followed by age and APACHE II. SOFA score, platelet count, and procalcitonin were less informative, and d-dimers showed no impact.

Limitations and Future Work

  • Small Sample Size: The authors acknowledge that the number of cases was small, and the created model is considered preliminary. They plan to extend the database to include more patients and different types of sepsis (bacterial and viral).
  • Potential Overfitting: Due to the small dataset, there's a risk of the machine learning model being overfitted. Future studies with larger datasets are needed to address this.
  • Interpretability of Complex Tasks: While XAI was used effectively, the authors suggest future work could explore interpretability for even more sophisticated AI tasks.
  • Continuous Improvement: The web-based application is designed for continuous updates, allowing the model to learn and improve as more data becomes available. This iterative approach is key to refining the survival prediction tool.

Summary updated

Abstract

Fibronectin (FN) plays an essential role in the host's response to infection. In previous studies, a significant decrease in the FN level was observed in sepsis; however, it has not been clearly elucidated how this parameter affects the patient's survival. To better understand the relationship between FN and survival, we utilized innovative approaches from the field of explainable machine learning, including local explanations (Break Down, Shapley Additive Values, Ceteris Paribus), to understand the contribution of FN to predicting individual patient survival. The methodology provides new opportunities to personalize informative predictions for patients. The results showed that the most important indicators for predicting survival in sepsis were INR, FN, age, and the APACHE II score. ROC curve analysis showed that the model's successful classification rate was 0.92, its sensitivity was 0.92, its positive predictive value was 0.76, and its accuracy was 0.79. To illustrate these possibilities, we have developed and shared a web-based risk calculator for exploring individual patient risk. The web application can be continuously updated with new data in order to further improve the model.
BibTeX
@inproceedings{lemanskaperek2022explainable,
  author = {Anna Lemańska-Perek and Dorota Krzyżanowska‐Gołąb and Katarzyna Kobylińska and Przemysław Biecek and Tomasz Skalec and Maciej Tyszko and Waldemar Goździk and Barbara Adamik},
  title = {{Explainable Artificial Intelligence Helps in Understanding the Effect of Fibronectin on Survival of Sepsis}},
  booktitle = {Cells},
  year = {2022},
  doi = {10.3390/cells11152433},
}
Figure