A New Strategy for Providing Definitions In Task-Oriented Dialogues

Margaret H. Sarner, Sandra CarberryCOLING 1988

TL;DR

This paper introduces a novel strategy for generating definitions in task-oriented dialogues, moving beyond static, generic responses. The core idea is to dynamically tailor definitions based on a sophisticated model of the user's knowledge, goals, and receptivity, ensuring maximum helpfulness and relevance. This approach prioritizes content selection based on user needs, guiding the choice of rhetorical strategy rather than the other way around.

What Problem Does This Paper Address?

In task-oriented dialogues, like those between an expert and a user trying to accomplish a specific goal (e.g., a user asking a tax agent for advice), the expert often needs to define terms. Traditional definition systems, or even human experts responding to isolated queries, tend to provide generic definitions. However, in an ongoing dialogue, the expert has a unique opportunity to be far more helpful by understanding why the user needs the definition and what they already know. The problem is how to leverage this contextual information to generate definitions that are not just accurate but also maximally useful and tailored to the individual user's current needs and knowledge state. This involves going beyond simply stating facts to providing information that directly aids the user in achieving their immediate goals.

Methods

The paper proposes a strategy centered on a dynamically inferred model of the user. This model comprises:

  • User's Domain Knowledge: Tracked by marking pieces of the knowledge base the user is familiar with.
  • User's Task-Related Plans and Goals: Represented by a context tree, indicating the user's current focus of attention within their overall plan.
  • User's Receptivity to Information: Modeled by assigning weights to different "strategic predicates" (types of definitional components).

The core of the strategy involves:

  1. Predicate Weighting: Initially, a local predicate receptivity model is created based on global weights, then adjusted based on the specific definition request and the user's inferred domain expertise. For instance, if a user asks "What is X?" where X is an object, the "Identification" predicate is weighted higher.
  2. Proposition Weighting: Predicates are filled with relevant information (propositions) from the knowledge base. The significance of each proposition is calculated by combining two factors:
  3. Relevance: How close the proposition's source is to the user's current focus of attention in their plan, modeled by an inverse exponential function based on "focus domain shifts."
    • Familiarity: The system's belief in the user's existing knowledge of the proposition's content, modeled by a function that drops off rapidly as familiarity decreases.
  4. Significance: A weighted combination of relevance and familiarity, with relevance weighted twice as heavily.
  5. Content Selection: Propositions are ranked by significance and categorized into "Must Say," "Say if Convenient," "Say if Needed for Coherence," and "Do Not Say." An answer generator then constructs an utterance using these categories, prioritizing "Must Say" and "Say if Convenient" propositions, while adhering to common rhetorical practices.

This approach contrasts with previous work (like McKeown's TEXT system) which treated definition requests as isolated queries and selected rhetorical strategies first, then content. This paper argues that content should drive strategy.

Datasets

Not discussed in the paper. The authors mention analyzing "transcripts of novice-expert dialogues from a variety of domains" for their study, but do not specify formal datasets or benchmarks.

Key Insights and Ideas

  • Content Dictates Strategy: The most crucial insight is the inversion of the traditional approach: the content deemed most important for the user should guide the rhetorical strategy used to present it, not the other way around.
  • Dynamic User Modeling is Key: Effective definitions in dialogues depend on a rich, dynamic understanding of the user's knowledge, current goals, and plans. This goes far beyond static user profiles.
  • Principle of Usefulness: Definitions should be at a level that is both understandable (not too complex) and informative (not stating the obvious or irrelevant). This principle guides the selection of appropriate content.
  • Contextual Relevance: The relevance of information is not absolute but depends heavily on the user's immediate task context and focus of attention. Information closer to the user's current goal is more relevant.
  • Familiarity Matters: Even highly relevant information is useless if the user doesn't understand it. The system must account for what the user likely already knows.
  • Strategic Predicates: Defining entities involves components that relate conceptual units (e.g., Identification, Properties, Operation), and the selection of which predicates to use and how to fill them is crucial.

Key Results

The paper presents a strategy and a framework for generating definitions, rather than empirical results from a deployed system. The "results" are the proposed methods for weighting predicates and propositions and the Principle of Usefulness. The authors illustrate their strategy with examples demonstrating how their approach would produce more tailored definitions than a generic system, such as distinguishing between a definition of "amaretti" for someone selling them internationally versus someone using them as an ingredient.

Limitations and Future Work

  • Limitations:
    • The current relevance metric treats all shifts among focus domains equally, which the authors acknowledge might not be optimal. Information from higher-level plans might be more relevant than from deeply nested subplans, even if equidistant.
  • The paper does not present empirical results from a deployed system.
  • Future Work:
    • Investigate strategies for dynamically weighting strategic predicates based on user receptivity.
  • Develop an "answer generator" that combines weighted propositions to produce coherent utterances, prioritizing higher-weighted ones.
  • Implement and test the proposed definition strategy to demonstrate its utility.
  • Further research into the optimal way to combine relevance and familiarity metrics.

Summary updated

BibTeX
@inproceedings{sarner1988new,
  author = {Margaret H. Sarner and Sandra Carberry},
  title = {{A New Strategy for Providing Definitions In Task-Oriented Dialogues}},
  booktitle = {International Conference on Computational Linguistics},
  year = {1988},
}
Figure