On the Underspecification of Situations in Open-domain Conversational Datasets

Naoki Otani, Jun Araki, HyeongSik Kim, Eduard HovyACL 2023

TL;DR

This paper argues that current open-domain conversational datasets lack crucial situational information, leading to underspecified problem spaces and suboptimal dialogue system behavior. The authors demonstrate that explicitly providing situational context can improve response coherence and specificity, but also highlight the challenge of distinguishing relevant from irrelevant information.

What Problem Does This Paper Address?

Modern conversational AI systems, like chatbots and virtual assistants, are increasingly sophisticated. A key driver of this progress has been the development of large datasets of conversations, which allow these systems to learn from vast amounts of human interaction. However, the authors of this paper point out a critical flaw in many of these datasets: they often omit information about the situation in which the conversation is taking place. This "underspecification" means that crucial context – like the presence of specific objects, the relationships between people, or the environment – is missing. Without this situational grounding, conversational systems are forced to make assumptions, which can lead to responses that are irrelevant, nonsensical, or even contradictory. This makes it difficult to develop and evaluate truly effective conversational agents, as their behavior is not reliably tied to the real-world context they are supposed to operate within.

Methods

The paper tackles this problem through a combination of analysis and empirical evaluation.

  1. Analysis of Existing Datasets: The authors first review the landscape of open-domain conversational datasets, highlighting that most rely solely on dialogue history and lack explicit situational information.
  2. Empirical Evaluation: To investigate the impact of situational information, they conduct experiments using three English datasets: SUGAR (which provides rich situational information), CICERO, and ConvAI2 (which do not).
  3. Situational Statement Generation: For datasets lacking situational context, they employ a fine-tuned COMET DIS model (based on GPT2-XL) to automatically generate situational statements. These statements cover categories like date, time, location, behavior, environment, and possession.
  4. Response Generation Models: They use two baseline systems for response generation: BlenderBot2 (BB2) and GPT-3. These models are tasked with generating responses both with and without the provided situational information.
  5. Manual Evaluation: Human annotators evaluate the generated responses based on three criteria: grammaticality, coherence, and context-specificity. They also conduct a study on GPT-3's robustness to irrelevant situational statements (distractors).

Datasets

The paper utilizes the following datasets:

  • SUGAR: A dataset designed for help-seeking scenarios, notable for providing 12 sentences of situational information per example, including distractors.
  • CICERO: A compilation of three datasets (DailyDialog, MuTual, DREAM) that do not explicitly present situational information.
  • ConvAI2: A dataset designed for persona-based conversations, which includes user-based features but lacks explicit situational context.

The authors also generated situational statements for CICERO and ConvAI2 to create situated versions of these datasets for their experiments.

Key Insights and Ideas

  • Situational Information is Crucial for Coherence: The most striking insight is that providing explicit situational context significantly improves the coherence and specificity of generated responses. This suggests that current models are "underspecified" without this grounding.
  • Distinguishing Relevant from Irrelevant Information is a Major Challenge: While situational information helps, models can be easily misled by irrelevant statements (distractors). This was particularly evident in GPT-3's performance, where coherence scores dropped as the number of distractors increased.
  • Models Make Implicit Assumptions: In the absence of explicit situational information, conversational models are forced to make assumptions about the world state. These assumptions can be incorrect and lead to undesirable behavior, essentially a form of "hallucination."
  • Automatic Generation of Situational Data is Promising but Imperfect: The paper demonstrates that automatically generating situational statements is feasible, but manual revision was necessary to ensure quality and remove conflicting facts. This points to a potential avenue for enriching existing datasets.
  • Different Models Have Different Strengths/Weaknesses: GPT-3 generally produced more fluent responses, even without situational input, suggesting it might have learned robust internal assumptions. BlenderBot2, however, showed a clearer improvement in coherence and specificity when situational information was provided.

Key Results

  • Improved Response Quality with Situational Input: Across datasets and models, providing situational information generally led to higher scores for coherence and context-specificity. For instance, in Figure 1, both BB2 and GPT-3 showed noticeable gains in these metrics when situational information was present.
  • GPT-3's Coherence Suffers with Distractors: While GPT-3 performed well overall, its coherence scores decreased significantly as the number of irrelevant situational statements increased (from 0.86 with no distractors to 0.77 with 10 distractors, as shown in Figure 2).
  • BlenderBot2 Benefits More Directly: BB2 showed a more pronounced improvement in coherence and specificity when situational information was provided, suggesting it relies more heavily on explicit context.
  • Inter-Annotator Agreement Challenges: The study noted relatively low inter-annotator agreement (Fleiss' kappa of 0.38) for manual evaluation, particularly for BB2, likely due to the subjective nature of the task and the fluency of responses.

Limitations and Future Work

  • Determining the "Adequate Amount" of Situational Information: The paper acknowledges the difficulty in defining how much situational information is "enough" and the challenge of the "frame problem" – determining which aspects of a situation are relevant and change over time.
  • Scalability and Hyperparameter Tuning: The experiments were conducted on a limited scale, and the authors suggest that larger models and more extensive hyperparameter tuning could yield different results.
  • Bias and Safety: The authors explicitly state that incorporating situational information does not inherently address issues of bias and safety in conversational AI and could potentially amplify existing problems if not handled carefully.
  • Future Work: The paper suggests exploring methods to automatically induce and represent situational information more effectively, developing robust techniques for identifying relevant context, and investigating how to better integrate diverse forms of situational information (e.g., images, audio). They also emphasize the need for careful monitoring and debiasing techniques to mitigate potential negative impacts.

Summary updated

BibTeX
@inproceedings{otani2023underspecification,
  author = {Naoki Otani and Jun Araki and HyeongSik Kim and Eduard Hovy},
  title = {{On the Underspecification of Situations in Open-domain Conversational Datasets}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2023},
}
Figure