Explanation Structures in XSEL

Karen KukichACL 1985

TL;DR

This paper introduces "link-dependent message generation" as a novel technique for creating accurate and direct explanations in expert systems. By leveraging structured knowledge representations called "links" within the XSEL system, it bridges the gap between an expert system's reasoning process and natural language explanations, offering a more coherent and informative user experience.

What Problem Does This Paper Address?

Expert systems, particularly those dealing with complex decision-making like configuring computer systems, often need to explain their reasoning to users. Traditional explanation methods, such as canned text or execution traces, suffer from significant drawbacks. Canned text explanations can become inaccurate when the underlying system logic changes, while execution traces can be overly detailed, including "computer artifacts" that are irrelevant to the user's understanding. The challenge is to generate explanations that are both accurate (reflecting the system's true reasoning) and direct (focusing on what the user needs to know without extraneous detail).

Methods

The paper proposes a "link-dependent message generation" approach, a core component of the XSEL explanation generator. XSEL itself is an expert system that assists salespeople in configuring VAX computer systems. The explanation generator operates in two phases: first, it organizes messages based on the system's working memory, and then it maps these messages into natural language.

The key innovation is the use of "links" – simple knowledge structures that represent relationships between elements in XSEL's working memory. Two types of links are central:

  • Generic Links: These are unnamed, binary links representing a direct dependency between a fact and its inferred value. They are easy to create but provide less explanatory power.
  • Relational Links: These are named, n-ary links that explicitly capture the type of relationship (e.g., arithmetic, satisfaction, allocation) between facts. These are more complex to create but are crucial for generating direct and informative explanations.

The explanation generator uses these links to build two dynamically growing networks: a generic network and a relational network. For "why-fact" and "why-line-item" queries, a "link-dependent message generator" examines these links to determine the relationships between facts and construct appropriate messages. The system also employs a queue-based mechanism to manage the order of explanations and allows users to control the depth of explanation ("explain more"/"explain less").

Datasets

Not discussed in the paper. The paper focuses on the internal workings of the XSEL system and its explanation generator, rather than external datasets for training or evaluation.

Key Insights and Ideas

  • Links as the Bridge: The core insight is that the relationships (links) captured by an expert system's inference rules can directly inform the structure and coherence of natural language explanations.
  • Two Networks for Two Needs: The creation of both a generic dependency network and a richer relational network allows for a trade-off between ease of creation and explanatory power. Relational links are essential for direct explanations.
  • Controlled Explanation Depth: The "explain more"/"explain less" functionality, managed by a queue, provides users with control over the level of detail, making explanations more adaptable to user needs.
  • Generic vs. Relational Explanations: The distinction between generic explanations (simple dependency statements) and relational explanations (explicating the nature of the relationship) highlights how different types of links lead to different qualities of explanation.

Key Results

The paper demonstrates the effectiveness of its link-dependent message generation approach through sample explanations for three query types: "why-choice" (why a component was selected), "why-fact" (why a specific fact has a certain value), and "why-line-item" (why a quantity is needed for an item). The generated explanations are presented as being both accurate (originating from working memory) and direct (focusing on relevant relationships). The paper also notes that the relational network, formed by abstract links, leads to sparser networks and more direct explanations.

Limitations and Future Work

  • Manual Creation of Relational Links: While generic links and arithmetic relational links can be largely automated, abstract relational links currently require manual intervention. This is a significant limitation for scaling the system.
  • Domain-Specific Relations: The current system has identified only twenty-six different reasoning relations in XSEL. The long-term goal is to identify more relations and potentially discover domain-independent sets of reasoning relations and schemas.
  • Potential for Knowledge Acquisition: The authors suggest that a domain-independent set of reasoning relations and schemas could facilitate the design of knowledge acquisition systems that elicit knowledge and generate inference rules.
  • Comparative Message Generator: The paper mentions a "comparative message generator" for "why-choice" queries but reserves a detailed discussion for future work.

Summary updated

BibTeX
@inproceedings{kukich1985explanation,
  author = {Karen Kukich},
  title = {{Explanation Structures in XSEL}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {1985},
}
Figure