Evaluating the Effectiveness of Explainable Artificial Intelligence Approaches (Student Abstract)

Jinsun Jung, Hyeoneui KimAAAI 2024 · DOI

TL;DR

This paper investigates how users perceive the effectiveness of different Explainable Artificial Intelligence (XAI) methods in a healthcare context. The study reveals that the intuitiveness of an explanation significantly impacts its perceived interpretability and plausibility, suggesting that simpler, more understandable XAI outputs might lead to greater user acceptance.

What Problem Does This Paper Address?

As Artificial Intelligence (AI), particularly deep learning, becomes more prevalent in critical fields like healthcare, understanding why an AI makes a certain prediction is crucial. This is especially true given regulations like GDPR, which grant individuals the "right to an explanation." While many XAI techniques exist, there's a lack of standardized methods to evaluate their effectiveness. This paper addresses this gap by exploring how users interpret and accept different XAI explanations, aiming to identify factors that influence this acceptance.

Methods

The researchers used a real-world dataset from the Korean National Health and Nutrition Examination Survey (KNHANES) to build an obesity prediction model using XGBoost. Four common XAI techniques were then applied to explain the model's predictions:

  • XGBoost Feature Importance: Identifies which features contributed most to the prediction.
  • SHapley Additive exPlanations (SHAP): A game-theory approach to attribute feature contributions to individual predictions.
  • Local Interpretable Model-agnostic Explanations (LIME): Explains individual predictions by approximating the complex model locally with an interpretable one.
  • Permutation Importance: Measures the decrease in model performance when a feature's values are randomly shuffled.

To evaluate these explanations, a focus group of seven graduate students in health science research was convened. They engaged in semi-structured interviews and open discussions to assess the interpretability and plausibility of the XAI outputs. The process involved manually ranking risk factors, evaluating XAI approaches, ranking the generated results by plausibility, and sharing opinions.

Datasets

The study utilized the Korean National Health and Nutrition Examination Survey (KNHANES) dataset. This dataset comprised 24,798 individuals over 19 years old and included 94 variables. A binary outcome variable was created based on a Body Mass Index (BMI) threshold of 25 kg/m².

Key Insights and Ideas

  • Intuitiveness is King: The study strongly suggests that the intuitiveness of an XAI explanation is the most significant factor influencing its perceived interpretability. If an explanation is easy to grasp, users are more likely to find it understandable.
  • Interpretability Fuels Plausibility: There's a clear link between how interpretable an explanation is and how plausible users find it. This implies that making XAI outputs understandable can directly increase user trust in the AI's results.
  • Beyond "Why": While not a primary focus, the idea of "why not" explanations (contrastive explanations) was mentioned as potentially helpful for deeper understanding.
  • Information Overload is Bad: Too many predictors presented as explanations can overwhelm users and hinder understanding.

Key Results

The focus group ranked the XAI approaches based on subjective interpretability and perceived plausibility. A lower average score indicates a higher ranking.

  • XGBoost Feature Importance was rated as the most plausible XAI explanation.
  • SHAP and LIME followed, with similar plausibility ratings.
  • Permutation Importance received the lowest plausibility rating.
  • Interpretability and plausibility rankings were largely consistent across methods, with SHAP being a slight exception.
  • The most frequently cited reasons for ranking XAI approaches included intuitiveness, scope of explanation, mechanism of explanation, and contrastive explanation.

Limitations and Future Work

The authors acknowledge that the generalizability of their findings is limited due to the small scope of the study and the relatively small focus group size. They suggest that a large-scale study with a larger focus group is necessary to establish a standardized framework for evaluating the effectiveness of XAI explanations. Future work could also explore the impact of different user backgrounds and expertise on their perception of XAI.

Summary updated

BibTeX
@inproceedings{jung2024evaluating,
  author = {Jinsun Jung and Hyeoneui Kim},
  title = {{Evaluating the Effectiveness of Explainable Artificial Intelligence Approaches (Student Abstract)}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2024},
  doi = {10.1609/aaai.v38i21.30458},
}
Figure