Tailoring Explanations for the User

Kathleen R. McKeown, Myron Wish, Kevin Matthews

This 1985 paper from Columbia University and AT&T Bell Laboratories represents a foundational contribution to Natural Language Generation (NLG) and User Modeling. It addresses a persistent challenge in Artificial Intelligence: the development of expert systems capable of providing explanations that are not only technically accurate but also contextually relevant to the individual user.

The Challenge of Expert Explanations

In the mid-1980s, while expert systems were increasingly proficient at problem-solving, their explanatory capabilities remained limited. Most systems offered a "one-size-fits-all" trace of internal logic, which was frequently too technical or irrelevant to the user's specific needs.

The authors argue that an effective explanation must align with the user's perspective. For instance, a student inquiring about a specific course might be motivated by a desire to graduate quickly, or by a deep interest in a particular subfield like Artificial Intelligence. A system that fails to recognize these differing goals will provide justifications that miss the mark. The research addresses this gap by enabling systems to infer shifting user goals and adjust their communicative strategies accordingly.

Methodology: The Advisor System

The researchers developed a prototype "Advisor" system for computer science students, built upon three primary architectural components:

Intersecting Knowledge Hierarchies

Rather than employing a single, flat database, the system organizes domain knowledge into multiple, overlapping hierarchies. This allows a single entity, such as a "Data Structures" course, to exist simultaneously within several contexts: a Requirements hierarchy, a State Model hierarchy (governing timing and sequence), and a Personal Interests hierarchy.

Partial state model and requirements hierarchies for the Advisor system.

Goal Inference

Building on the goal-inference frameworks of Allen and Perrault, the system tracks a sequence of user questions to identify an underlying "relevant goal." For example, if a student asks about "Natural Language Processing" followed by "AI," the system identifies the lowest common ancestor in a plan tree—in this case, the goal of "Concentrating in AI."

A goal tree showing the inference of a relevant user goal.

Goal-Driven Rule Firing

The system utilizes a production system where inferred goals act as a filter. Once a goal is identified, the system "partitions" the knowledge base, loading only the facts relevant to that specific point of view into its working memory. This ensures that the generated explanation is derived only from rules and data that match the user's perspective.

Key Insights and Ideas

  • Point of View (POV) as a Functional Filter: A central innovation is the treatment of "perspective" as a mechanism for knowledge partitioning. By restricting the system's focus to a specific hierarchy, the resulting explanation is naturally tailored because irrelevant justifications are effectively hidden from the generation process.
  • The "Likelihood" Scale: To avoid making incorrect assumptions, the authors categorized inferred goals into three levels: Definite (explicitly stated), Likely (repeatedly hinted at), and Plausible (inferred from a single query). Tailored explanations are only provided for "Definite" or "Likely" goals; otherwise, the system defaults to a neutral response.
  • Discourse-Level Analysis: The paper emphasizes the importance of analyzing sequences of questions rather than isolated utterances. This longitudinal view allows the system to build a more robust and accurate model of user intent over time.

Results and Proof of Concept

The paper provides a qualitative proof of concept through the "Advisor" prototype. For a single question—"Should I take both discrete math and data structures this semester?"—the system demonstrated the ability to generate distinct, natural language justifications based on the inferred POV:

  • Requirement POV: "Yes, data structures is a requirement for all later CS courses..."
  • State Model POV: "Yes, you usually take them both first semester, sophomore year..."
  • Scheduling POV: "Yes, they're offered next semester, but not in the spring..."

The system successfully integrated an ATN parser, a goal inferencer, and a surface generator to produce these tailored responses in real-time.

Limitations and Future Directions

Despite its successes, the research highlighted several areas for future work:

  • Knowledge Engineering: The intersecting hierarchies and plan trees required intensive manual construction, posing a challenge for scaling to larger domains.
  • Static vs. Dynamic Models: The system focused on dynamic, immediate goals but lacked a model for static user traits, such as prior knowledge or academic history.
  • Discourse Strategy: While the system could select a perspective, it lacked sophisticated strategies for structuring the overall flow and organization of the explanation.

At the time of publication, certain features, such as the "Likely" goal classification and advanced surface generation, were still in the development phase.

Summary updated

BibTeX
@inproceedings{mckeown1985tailoring,
  author = {Kathleen R. McKeown and Myron Wish and Kevin Matthews},
  title = {{Tailoring Explanations for the User}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {1985},
  url = {https://openreview.net/forum?id=SyOnXM_Wr},
}
Figure