Towards an Explainer-agnostic Conversational XAI

Navid Nobani, Fabio Mercorio, Mario MezzanzanicaIJCAI 2021 · DOI

TL;DR

This paper introduces a novel framework for Explainable AI (XAI) that aims to be agnostic to the underlying black-box model and the specific explanation generation method. By focusing on a conversational interface and user interaction, they propose a more flexible and adaptable XAI system that can cater to diverse user needs and technical setups.

What Problem Does This Paper Address?

The field of Explainable AI (XAI) is rapidly growing, driven by the increasing complexity of machine learning models. While XAI aims to make these "black-box" models more transparent, current approaches often fall short. Many XAI systems rely on non-language-based explanations (like feature lists or simple graphics) or basic Natural Language Generation (NLG) techniques that are rigid and fail to build user trust. Furthermore, existing XAI systems are often tightly coupled to specific black-box models and explanation types, limiting their adaptability. This paper addresses the need for XAI systems that are more user-centric, flexible, and can engage in natural, interactive conversations to provide explanations, regardless of the underlying technical components.

Methods

The core of the proposed approach is an explainer-agnostic conversational XAI framework. This framework is designed to decouple the data, the black-box model, and the explainer from the conversational interface. The key innovation lies in a graph-based representation that maps the connections between these system elements. This allows for easier adaptation to new datasets, black-box models, and explainer types.

The conversational aspect is handled by a dialogue system built using the RASA framework. This system includes:

  • Natural Language Understanding (NLU): To interpret user queries.
  • Dialogue State Tracker: To manage the conversation's context.
  • Dialogue Policy: To decide the system's next action.
  • Natural Language Generation (NLG): To formulate explanations.

Crucially, the dialogue system's components are trained based on the attached data, black-box, and explainer, rather than being hardcoded. This flexibility extends to explanation generation, where they aim to use State-of-the-Art (SOTA) models for both plain-fact and contrastive explanations, moving beyond simple template filling. The system also supports generating explanations in both textual and graphical forms, including PDP and ICE plots.

Datasets

Not discussed in the paper. The paper focuses on the framework's design and methodology rather than specific empirical evaluations on benchmark datasets.

Key Insights and Ideas

  • Explainer Agnosticism is Key: Decoupling the explanation generation from the conversational interface is crucial for building adaptable XAI systems. This allows for swapping out different explainers without redesigning the entire dialogue system.
  • Conversational XAI for Deeper Understanding: Moving beyond static explanations to an interactive dialogue allows users to ask follow-up questions, clarify terms, and explore explanations based on their specific context and prior knowledge. This is vital for building trust and enabling informed decision-making.
  • User-Centricity is Paramount: The authors emphasize that XAI systems should be designed with the end-user in mind, considering their needs, prior knowledge, and how they interact with the system. This contrasts with many current XAI systems that are technically sophisticated but inaccessible to non-experts.
  • Flexibility through Graph Representation: Using a graph to map system components facilitates the adoption of the framework to new scenarios, datasets, and black-box models, reducing the effort required for adaptation.

Key Results

The paper presents a conceptual framework and a proposed system architecture. While specific quantitative results are not detailed, the authors highlight the potential for their system to:

  • Facilitate the adoption of XAI to new cases (e.g., new datasets).
  • Achieve a solution that is explainer-agnostic and not bounded to a specific data type or black-box.
  • Provide interactive systems that allow end-users to directly interact with explanations.

Limitations and Future Work

The authors acknowledge that they are currently working on several elements:

  • Multi-turn open-domain dialogue management: Enhancing the system's ability to handle complex, extended conversations, including cohesion, coherence, and ellipsis resolution.
  • User study: Conducting a user study to evaluate the efficacy of the system in facilitating the explanation process.
  • Data augmentation: Using Amazon Mechanical Turk to enhance the NLU training data by increasing the variety of possible user queries.

The paper also implicitly suggests that a limitation of current XAI is their static nature and lack of natural language interaction, which their proposed system aims to overcome.

Summary updated

BibTeX
@inproceedings{nobani2021towards,
  author = {Navid Nobani and Fabio Mercorio and Mario Mezzanzanica},
  title = {{Towards an Explainer-agnostic Conversational XAI}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2021},
  doi = {10.24963/ijcai.2021/686},
}
Figure