Exploiting User Expertise in Answer Expression

David N. ChinAAAI 1988

Introduction

In the late 1980s, technical help systems faced a significant usability hurdle: they were largely "one-size-fits-all." A seasoned system administrator and a total novice would receive the same verbose explanation for a command. This lack of personalization led to frustration for experts, who found the redundant information distracting, and confusion for novices, who often lacked the prerequisite context to understand dense technical jargon.

To address this, David Chin developed UCExpress, a component of the UNIX Consultant (UC) system designed to tailor natural language answers based on a user's inferred expertise.

The Challenge of Answer Expression

The core problem UCExpress tackles is answer expression—the pragmatic process of deciding not just what information is correct, but how to present it effectively. This involves considering the user's prior knowledge and the current conversational context to ensure the response is neither too sparse nor overly redundant.

UCExpress: Pruning and Formatting

UCExpress operates on a conceptual network using the KODIAK representation language. The system processes a planned answer through two distinct phases:

  1. Pruning: The system identifies sub-concepts within a planned answer that the user likely already knows. By marking these for pruning, the natural language generator can skip redundant facts, leading to more concise responses.
  2. Formatting: Once the essential information is identified, UCExpress selects an "expository format" to improve clarity. For example, if a user is unfamiliar with a command's syntax, the system might provide a concrete example (e.g., "type rm foo"). Alternatively, if the user knows a related command, it might use a simile (e.g., "ruptime is like uptime").
A KODIAK conceptual network representing the internal logic for a print command.

Modeling Expertise with KNOME

The intelligence behind UCExpress's pruning decisions comes from KNOME (KNOledge Model of Expertise). KNOME categorizes both users and facts into hierarchical levels. Users are assigned a "stereotype" (Novice, Beginner, Intermediate, or Expert), while UNIX facts are categorized by difficulty (Simple, Mundane, Complex, or Esoteric).

The KNOME stereotype model mapping user levels to knowledge difficulty.

By mapping these two hierarchies, the system can make computationally efficient inferences. For instance, an "Intermediate" user is assumed to know all "Simple" and "Mundane" facts. If a planned answer contains a "Mundane" fact, UCExpress marks it for pruning for an intermediate user but retains it for a novice.

Key Insights

  • Pruning vs. Deletion: Pruned concepts are not deleted from the underlying representation. They remain available for linguistic tasks like generating pronouns (anaphora) or shortened phrases (ellipsis), ensuring the final output remains grammatically coherent.
  • Similes as Compression: The paper highlights that similes are more than just "friendly" explanations; they are efficient compression tools. Explaining a new command via comparison is often shorter and clearer than a full definition, provided the user has the necessary background knowledge.
  • Separation of Concerns: By decoupling the "planner" (which finds the answer) from UCExpress (which decides how to say it), the system separates domain logic from pragmatic communication strategies.

Results and Limitations

The paper demonstrates qualitative improvements in system responses. For novices, the system provides full command formats and concrete examples. For intermediate and expert users, it produces succinct answers, often omitting standard file-command formats that are considered common knowledge at those levels.

However, the system faces limitations. Pragmatic pruning can sometimes conflict with grammatical rules, potentially creating ambiguity if a pronoun is used where a full noun phrase was necessary for clarity. Additionally, the reliance on stereotypes means the system may occasionally prune too much or too little if a specific user's knowledge deviates from the general profile.

Conclusion

UCExpress represents an early and influential attempt to move beyond static help systems. By modeling user expertise and applying pragmatic constraints to answer generation, it paved the way for more sophisticated, context-aware digital assistants.

Summary updated

BibTeX
@inproceedings{chin1988exploiting,
  author = {David N. Chin},
  title = {{Exploiting User Expertise in Answer Expression}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {1988},
}
Figure