Description Strategies for Naive and Expert Users

Cecile L. ParisACL 1985

TL;DR

This paper investigates how to tailor natural language descriptions of complex physical objects to users' varying levels of expertise. The authors found that adult encyclopedias describe objects by their sub-parts and properties, while junior encyclopedias focus on the functional process. This distinction is crucial for developing more effective question-answering systems.

What Problem Does This Paper Address?

Question-answering systems are most useful when they can adapt their responses to individual users. One key aspect of this adaptation is tailoring the level of detail and the type of information presented based on the user's knowledge about the domain. For instance, explaining a car engine to a child requires a different approach than explaining it to a mechanic. This paper tackles the challenge of identifying and implementing these different descriptive strategies for "naive" (less domain knowledge) and "expert" (more domain knowledge) users, specifically in the context of describing complex physical objects.

Methods

The authors analyzed texts from adult and junior encyclopedias, as well as high school physics textbooks, to identify distinct descriptive strategies. They employed methods for decomposing paragraphs into primitive rhetorical structures, drawing on work by researchers like Hobbs, Mann, and McKeown. The core of their analysis involved classifying sentences based on rhetorical predicates and identifying recurring patterns or "schemas."

For adult descriptions, they found a prevalent "constituency schema," where objects are described in terms of their sub-parts and the properties of those parts. For junior descriptions, a "process trace" strategy emerged, where the description follows the sequence of events and causal relationships that enable the object to perform its function. This involves explaining "how" the object works step-by-step.

The paper also details the computational implementation of these strategies using a frame-based knowledge representation. This system uses the identified schemas (constituency for adults, process trace for juniors) to select and organize information from a database to generate descriptions.

Datasets

The primary "datasets" used in this study were:

  • Adult Encyclopedias: Texts describing complex physical objects.
  • Junior Encyclopedias: Texts describing complex physical objects, aimed at a younger audience.
  • High School Physics Textbooks: Texts also used for comparison.

The authors studied approximately fifteen examples from each category. They did not introduce a new benchmark dataset but rather analyzed existing texts to derive their descriptive strategies.

Key Insights and Ideas

  • Beyond Detail Level: The most significant insight is that tailoring descriptions isn't just about providing more or fewer details. The type of information presented differs fundamentally: adults get a structural breakdown (parts and properties), while juniors get a functional walkthrough (process and cause-effect).
  • Constituency vs. Process Trace: The identification of two distinct, dominant strategies – the "constituency schema" for experts and the "process trace" for novices – provides a concrete framework for generating tailored descriptions.
  • Functional Importance for Novices: Junior descriptions prioritize explaining the function of parts and how they contribute to the overall operation, introducing parts only when necessary for understanding the process.
  • Structural Importance for Experts: Adult descriptions prioritize enumerating sub-parts and their inherent properties, with functional explanations being secondary or brief.
  • Redundancy and Vividness in Junior Texts: Junior texts tend to be more redundant, reiterating steps for clarity, and often include more vivid, descriptive language (e.g., "little brass box filled with carbon of especially selected and treated coal") to aid comprehension.

Key Results

The paper's key results are qualitative findings about the descriptive strategies:

  • Adult Strategy: Primarily uses a "constituency schema" to describe objects by their sub-parts and their attributes.
  • Junior Strategy: Primarily uses a "process trace" to describe objects by following the causal chain of events that enable its function.
  • Computational Implementation: The authors demonstrated that these strategies can be implemented computationally, with their system generating descriptions based on the chosen schema (constituency for adults, process trace for juniors). Figures 3, 4, and 5 show printouts of these generated descriptions.

Limitations and Future Work

  • Limited Scope of Analysis: The study focused on a specific type of object description (physical objects) and a limited set of sources (encyclopedias, textbooks). The authors acknowledge that other domains or text types might exhibit different strategies.
  • Defining Expertise: The paper defines "naive" and "expert" based on domain knowledge, not system interaction experience. Determining a user's level of expertise is presented as a separate, ongoing research problem.
  • Arbitrary Databases: The current implementation relies on a test database where the order of relations is clear. For arbitrary databases, deciding where to begin describing a process and how to handle non-sequential processes will be more challenging.
  • Intermediate Expertise: The study focused on the two extremes (naive and expert). Future work needs to consider how to combine these strategies for users with intermediate levels of expertise.
  • Surface Generation: While the strategies dictate what information to include, the paper mentions ongoing work on augmenting an English surface generator to translate this information into coherent sentences, but the interface between strategies and generation remains to be fully studied.

Summary updated

BibTeX
@inproceedings{paris1985description,
  author = {Cecile L. Paris},
  title = {{Description Strategies for Naive and Expert Users}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {1985},
}
Figure