TL;DR
This paper introduces a novel framework for explainable AI (XAI) that allows users to interact with AI models in natural language. The system uses an And-Or Graph (AOG) model to understand input data and an XAI model to generate explanations for its predictions, focusing on how these explanations relate to user questions.
What Problem Does This Paper Address?
As AI models, particularly deep learning systems, become more complex, understanding why they make certain predictions is crucial. This is the domain of Explainable AI (XAI). However, current XAI methods often provide explanations in a format that is not easily interpretable or interactive for end-users. The challenge is to bridge the gap between AI's internal reasoning and a human's need for clear, contextualized explanations, especially when users want to probe specific aspects of a prediction or understand alternative scenarios.
Methods
The proposed framework has two main components:
- Prediction And-Or Graph (AOG) Model: This model is responsible for recognizing and localizing concepts within the input data (e.g., identifying actions and objects in a video). AOGs are hierarchical and compositional, allowing for the representation of complex contextual information and relationships.
- XAI Model for Natural Language Interaction: This component takes the AOG's predictions (represented as parse graphs) and generates explanations in natural language. The XAI model employs several types of reasoning:
- Direct Evidence: Using raw prediction scores (e.g., detection confidence).
- Part-Based Inference: Inferring a concept based on the presence and configuration of its detected parts.
- Spatiotemporal Context: Leveraging information about the surrounding environment and temporal sequence of events.
The core innovation lies in how the XAI model is designed to interact with users, specifically by categorizing user questions and tailoring explanations accordingly.
Datasets
The authors created a new explanation dataset by using the Youtube Action Videos dataset (Liu et al., 2009). This dataset contains videos from 11 action categories. For their dataset, they manually collected question-explanation pairs, with one question for each of their defined question types and five types of explanations for each question, totaling 550 pairs. This dataset was annotated by graduate and undergraduate students.
Key Insights and Ideas
- Question Categorization is Key: The paper proposes a detailed taxonomy of 10 question types (e.g., "Why X rather than not X?", "Why X rather than Y?", "What if X instead of Y?"). This categorization is crucial for understanding the user's intent and generating relevant explanations.
- Beyond Direct Evidence: While direct evidence (like detection scores) is a basic form of explanation, users often prefer explanations that leverage contextual information, part-based reasoning, and even counterfactuals or discourse relations.
- Discourse and Common Sense Reign Supreme: The experiments revealed that discourse-based explanations (how different parts of a video relate to each other) and common-sense explanations (integrating general world knowledge) were the most sought-after by users, even for simple questions. This highlights the need for AI to understand higher-level context and world knowledge.
- AOG's Foundational Role: The AOG model's ability to capture underlying evidence (Alpha, Beta, Gamma explanations) is still significant, accounting for over 30% of explanations, demonstrating its effectiveness in representing core visual concepts.
Key Results
The paper's primary contribution is the framework and the analysis of user preferences for different explanation types based on question types.
- User Preference for Context: For simple "WH-X" questions, users surprisingly preferred "AOG Gamma" explanations (top-down, context-based) over "AOG Alpha" (direct evidence).
- Discourse Dominance: "Discourse based explanations" were the most preferred for "DO-X-NOT-Y" questions.
- Common Sense Importance: "Common sense explanations" were highly preferred for "WH-NOT-Y", "NOT-X", "NOT-X1-BUT-X2", "NOT-X-BUT-Y", "DO-NOT-X", and "DO-X1-NOT-X2" question types.
- Table 1 provides a detailed breakdown of explanation type preferences across all question types, showing significant variation.
- Figure 2 visually summarizes that Discourse and Common Sense explanations are the most sought-after overall.
Limitations and Future Work
- Dialogue History: The authors acknowledge that dialogue or conversation history would be helpful in deciding between certain explanation types (e.g., Alpha vs. Gamma for WH-X-NOT-Y questions).
- Future Work: The authors plan to investigate the correlations between questions and explanation types by explicitly considering dialogue history (question-explanation pairs from previous turns). They also aim to integrate richer domain knowledge for better common-sense explanations and capture higher-level context through discourse relations.
Summary updated