Handling Uncertainty During Plan Recognition in Task-Oriented Consultation Systems

Bhavani Raskutti, Ingrid ZukermanUAI 1991

TL;DR

This paper introduces a novel approach to plan recognition in task-oriented systems that explicitly handles uncertainty arising from incomplete or inaccurate user input. By combining Bayesian probability theory with an information content measure, the system can effectively disambiguate multiple interpretations of user statements, leading to more robust and reliable assistance.

What Problem Does This Paper Address?

In task-oriented consultation systems (like a travel agent bot), the system needs to understand a user's goals and requirements to provide assistance. This is challenging because users often provide partial or ambiguous information. For example, a user might say "I want to go to Sydney," but not specify the departure date or origin. The system must infer the complete plan from these fragmented statements. Existing approaches often struggle when multiple interpretations are possible due to unreliable information sources (e.g., the user's knowledge, general world knowledge). This paper tackles the problem of selecting the most likely intended interpretation from a set of possibilities when faced with such uncertainty.

Methods

The core of the proposed mechanism involves three main processes:

  1. Direct Inference: This process generates an initial set of possible interpretations of user statements. It uses domain-specific knowledge (like action definitions) and discourse coherence rules. Uncertainty here arises from the inherent ambiguity of language and the multiple ways statements can relate to previous discourse. Bayesian probability theory is used to assign probabilities to these interpretations based on the likelihood of the statement, previous discourse, and the relation between them.
  2. Indirect Inference: Since direct inference often yields incomplete interpretations, this stage fills in the missing details using various information sources. These sources, ranked by reliability, include:
  3. User's Statements: Direct inferences from what the user explicitly says (highest reliability, strength 1).
  4. Domain Knowledge: Inferences based on the system's understanding of the specific domain (e.g., travel).
  5. Domain Assumptions: Inferences based on typical or normal behavior within the domain.
  6. User Model: Inferences based on a model of the specific user (if available).
  7. Common-Sense Knowledge: General world knowledge.

    The strength of an inference is directly proportional to the reliability of its source. The system prioritizes stronger inferences, preventing weaker ones from overriding more certain conclusions.

  8. Information Content Measure: This novel component quantifies how "well-defined" an interpretation is. It considers both the specificity of a parameter (how many possible values it can take) and the strength of the inference used to determine its value. An interpretation with high information content is considered more likely to be the intended one. This measure is used to:

  9. Prune interpretations: Interpretations with low probability relative to the most probable one are discarded based on a threshold.
  10. Update probabilities: The probabilities of interpretations are adjusted to favor those with higher information content.
  11. Determine stopping criteria: Processing stops when a complete interpretation (zero information content) is found or when no further increase in information content is possible.

Datasets

Not discussed in the paper. The authors mention using "five simple discourse samples" and "transcripts of telephone conversations at travel agencies" for testing, but these are not presented as formal datasets.

Key Insights and Ideas

  • Information Content as a Disambiguation Tool: The paper's central idea is that simply assigning probabilities to interpretations isn't enough. The quality or completeness of an interpretation (its information content) is a crucial factor in determining its likelihood. This is particularly relevant in cooperative settings where users aim to be understood.
  • Strength of Inference Matters: Explicitly modeling the reliability of information sources and assigning a "strength" to inferences is key to handling uncertainty. This prevents less reliable information from undermining more certain conclusions.
  • Bayesian Framework with Augmentation: The paper builds upon standard Bayesian approaches for probabilistic reasoning by adding the information content metric. This combination allows for a more nuanced evaluation of interpretations.
  • Domain Independence (mostly): The core mechanisms for probability assignment and information content calculation are designed to be domain-independent, relying on general principles of inference and discourse structure. Domain-specific knowledge is then used to refine these probabilities.

Key Results

The paper demonstrates the effectiveness of their approach through examples, showing how it correctly identifies the intended interpretation in plausible travel agency dialogues. For instance, in Example 1, the system correctly prioritizes an interpretation that involves a direct connection between the user's statements, even when multiple interpretations are initially possible. The information content measure helps to discard less likely interpretations that require postulating additional, unstated plans.

Limitations and Future Work

  • Computational Cost: The paper acknowledges that generating all possible interpretations can be computationally expensive, especially in complex domains. Their decision to consider only the strongest inference during indirect inference is a trade-off to manage this.
  • ICNORM Definition: The definition of ICNORM (a normalizing constant for information content) is noted as having a significant impact on probability updates, especially in restricted domains. The authors suggest that alternative definitions might be preferred in more realistic, open-ended domains.
  • Future Work: The authors suggest that their method could be extended to handle more complex domains and potentially be used for interpreting user statements in general, as well as in areas like multi-media document retrieval. They also mention experimenting with different definitions of ICNORM.

Summary updated

BibTeX
@inproceedings{raskutti1991handling,
  author = {Bhavani Raskutti and Ingrid Zukerman},
  title = {{Handling Uncertainty During Plan Recognition in Task-Oriented Consultation Systems}},
  booktitle = {Conference on Uncertainty in Artificial Intelligence},
  year = {1991},
}
Figure