Annotating Dialogue Acts to Construct Dialogue Systems for Consulting

Kiyonori Ohtake, Teruhisa Misu, Chiori Hori, Hideki Kashioka, Satoshi NakamuraAlr7@ijcnlp 2009

TL;DR

This paper introduces a new corpus of spontaneous consulting dialogues, the "Kyoto tour guide dialogue corpus," designed to train dialogue managers for more natural, decision-making systems. They propose a novel dialogue act annotation scheme that captures both the communicative function (speech act) and the semantic content of utterances, enabling more nuanced understanding of human interaction.

What Problem Does This Paper Address?

Building effective dialogue systems, especially for complex tasks like consulting where users might not have a clear objective, is challenging. Existing dialogue systems often rely on pre-defined scenarios or agendas, which are difficult to handcraft for spontaneous, human-like interactions. While large speech corpora exist, they often lack sufficient dialogue data or focus on very specific, limited tasks. The authors aim to bridge this gap by creating a rich dataset and annotation scheme specifically for consulting dialogues, enabling data-driven approaches like machine learning to build more robust dialogue managers.

Methods

The core of this paper is the development of a new dialogue act (DA) annotation scheme and the creation of a corresponding corpus. The DA annotation scheme is designed to capture two key aspects of an utterance:

  • Speech Act (SA) Tags: This layer describes the communicative function of an utterance. It extends existing tag sets like MRDA and DIT++ by adopting a policy of annotating with multiple labels per utterance to capture multifunctionality. The SA tags are organized into six layers: General, Response, Check, Constrain, ActionDiscussion, and Others, allowing for a detailed description of how an utterance contributes to the dialogue flow.
  • Semantic Content Tags: This layer focuses on what the utterance is about. It's based on a predicate-argument structure derived from dependency relations, which is considered more robust for fragmentary dialogue utterances than traditional syntax-based approaches. Semantic classes are assigned to these predicate-argument units, drawing parallels to named entity recognition.

The corpus itself is built from spontaneous tourist guidance dialogues in Japanese, collected in three modalities: face-to-face (F2F), Wizard of Oz (WOZ), and telephonic (TEL).

Datasets

The paper introduces the Kyoto tour guide dialogue corpus.

  • Size: 130 hours of consulting dialogues in the tourist guidance domain.
  • Breakdown:
    • 114 face-to-face (F2F) dialogues
    • 80 Wizard of Oz (WOZ) dialogues
    • 62 telephonic (TEL) dialogues
  • Content: Includes speech, transcripts, speech act tags, morphological analysis results, dependency analysis results, and semantic content tags.
  • Comparison: While large corpora like CSJ exist, the authors argue that the dialogue portion of CSJ is insufficient for statistical dialogue system training. Existing large dialogue corpora often cover very clear tasks in limited domains, whereas this corpus focuses on the more open-ended nature of consulting.

Key Insights and Ideas

  • Multifunctionality of Utterances: The decision to allow multiple speech act tags per utterance is crucial. Consulting dialogues are inherently complex, with utterances often serving several communicative purposes simultaneously. Capturing this richness is key to understanding and managing the dialogue.
  • Dialogue Flow in Consulting: The analysis of tag occurrence during dialogue episodes reveals distinct patterns. Early parts of decision-making episodes tend to involve information gathering (e.g., WH-Questions), while later parts focus on evaluation and commitment. This suggests a need for session or phase management in dialogue systems.
  • Predicate-Argument Structure for Semantics: Using dependency relations and predicate-argument structure for semantic content annotation is a practical approach for handling the often fragmented nature of spoken language in dialogues, making it more robust than purely syntax-based methods.
  • Hierarchical Semantic Classes: The proposed hierarchical semantic classes, with shared lower-level structures, offer reusability across different domains and tasks, making the annotation scheme more scalable.

Key Results

  • Annotation Scheme: A comprehensive dialogue act annotation scheme is proposed, covering both communicative function (speech act) and semantic content.
  • Corpus Collection: A substantial corpus of 130 hours of spontaneous consulting dialogues has been collected and is being annotated.
  • Reliability: Preliminary analysis shows good agreement among annotators for the speech act tags, with a Kappa statistic of 0.68 for all layers, indicating the annotation scheme is reliable.
  • Dialogue Tendencies: Analysis of tag occurrences revealed distinct patterns corresponding to different stages of the decision-making process in sightseeing planning dialogues.

Limitations and Future Work

  • Annotation Completion: The semantic content tag annotation is still in progress for a subset of the corpus (10 dialogues).
  • Future Work:
    • Constructing automatic taggers for speech act and semantic content tags using the annotated corpus and machine learning.
  • Condensing or selecting dialogue acts that directly influence dialogue flow to build a consulting dialogue system.
  • Adapting analyzers (morphological, dependency) for the domain.

Summary updated

BibTeX
@inproceedings{ohtake2009annotating,
  author = {Kiyonori Ohtake and Teruhisa Misu and Chiori Hori and Hideki Kashioka and Satoshi Nakamura},
  title = {{Annotating Dialogue Acts to Construct Dialogue Systems for Consulting}},
  booktitle = {Alr7@ijcnlp},
  year = {2009},
}
Figure