Visual Analytics for Guiding Feature Attribution Method Selection

Priscylla SilvaIJCAI 2025 · DOI

TL;DR

Choosing the right feature attribution method for explaining AI models is a major challenge due to conflicting explanations and a lack of universal best practices. This paper introduces "Visagreement," a visual analytics tool and methodology to help researchers understand and navigate these disagreements, and proposes a conceptual framework for selecting the most suitable explanation method.

What Problem Does This Paper Address?

Machine learning models are increasingly used in high-stakes domains like healthcare, making explainable AI (XAI) crucial for building trust and justifying decisions. Feature attribution methods are a key component of XAI, aiming to explain model predictions by assigning importance scores to input features. However, selecting the best feature attribution method for a given task is surprisingly difficult. This is because:

  • Conflicting Explanations: Different methods can produce contradictory explanations for the same prediction.
  • No One-Size-Fits-All: A method that works well for one model or dataset might not perform optimally for another.
  • Metric Mayhem: Evaluation metrics for explanations vary in what they measure, often leading to conflicting results.
  • Ad Hoc Choices: Practitioners often choose methods based on personal preference rather than systematic evaluation.

This paper tackles these challenges by developing tools and methodologies to systematically evaluate, compare, and select feature attribution methods, particularly for tabular data.

Methods

The research presents two main contributions:

  1. Visagreement: This is a visualization-assisted methodology designed to help researchers explore and understand disagreements between feature attribution methods. It consists of:
  2. A Visualization Tool: An interactive interface that allows users to explore the "disagreement space," identifying instances where methods agree or disagree.
  3. An Integrated Python Library: This library streamlines data processing and the execution of explanation methods, making it easier for users to apply their own models and datasets. Visagreement aims to facilitate hypothesis generation about why disagreements occur.

  4. Conceptual Framework for Method Selection: Based on a literature review, the authors propose a three-layer framework to guide practitioners in choosing feature attribution methods:

  5. Model-centric layer: Focuses on how well explanations reflect the model's behavior (e.g., fidelity).
  6. Explanation-centric layer: Assesses the quality of the explanation itself (e.g., robustness, interpretability).
  7. Human-centric layer: Evaluates the usefulness of the explanation for end-users (e.g., trust, clarity). The framework advocates for a bottom-up approach, ensuring explanations are faithful, then robust, and finally useful.

Datasets

The paper mentions the use of "seven datasets" for case studies within the Visagreement evaluation. Specific dataset names are not provided. The ongoing projects aim to study "more models and datasets" for robust results.

Key Insights and Ideas

  • Disagreement is Informative: While disagreements between attribution methods can be frustrating, they might actually highlight areas where the model is performing poorly or where explanations are less reliable.
  • Model Performance and Disagreement Link: The research found that instances where attribution methods disagreed the most often corresponded to instances where the model performed significantly worse. This suggests that high disagreement could be a red flag for model reliability.
  • Negative Class Bias: Disagreements were disproportionately higher for negative-class instances (label 0) in the studied models. This points to potential biases in how attribution methods explain predictions for certain classes.
  • No Simple Feature-Level Cause: Initial analyses did not identify specific features or subsets of features that consistently drive disagreements across different scenarios. This suggests the causes of disagreement are more complex than just individual feature importance.
  • Need for Holistic Selection: The literature review revealed a gap in tools that simultaneously support evaluation, comparison, and selection of explanation methods. The proposed framework aims to fill this by providing a structured, multi-layered approach.

Key Results

  • Visagreement Evaluation: Case studies using Visagreement revealed a strong correlation between model performance and the level of disagreement among feature attribution methods.
  • User Study for Selection Tool: A visual analytics Python library developed to aid in method selection was evaluated with 10 participants. They reported that the visualizations provided sufficient grounds for selecting methods for domain experts.
  • Framework Proposal: A conceptual framework with three layers (model-centric, explanation-centric, human-centric) was proposed to guide method selection.

Limitations and Future Work

  • Focus on Tabular Data: The current tools and research primarily focus on tabular data. Future work aims to adapt these approaches for image and text data.
  • Reducing Search Space: A key challenge in developing a comprehensive toolkit for method selection is reducing the vast search space of existing methods and metrics, making exhaustive evaluation impractical.
  • Resolving Metric Conflicts: Another challenge is resolving conflicts between different metrics within the same framework layer to ensure consistent method selection.
  • Further Model and Dataset Exploration: The authors plan to study more models and datasets to ensure robust results and generalizability.

Summary updated

BibTeX
@inproceedings{silva2025visual,
  author = {Priscylla Silva},
  title = {{Visual Analytics for Guiding Feature Attribution Method Selection}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2025},
  doi = {10.24963/ijcai.2025/1243},
}
Figure