Dialogue Discourse Parsing as Generation: A Sequence-to-Sequence LLM-based Approach

Chuyuan Li, Yuwei Yin, Giuseppe CareniniSIGDIAL 2024 · DOI

Abstract

Dialogue discourse parsing (DDP) involves mapping the logical structure of conversations by identifying relationships between elementary discourse units (EDUs). Traditionally, this has been approached as a structural extraction task requiring specialized architectures. We propose a paradigm shift, framing DDP as a sequence-to-sequence (Seq2Seq) text generation problem. By fine-tuning instruction-tuned Large Language Models (LLMs) like T0-3B, we demonstrate that generative models can effectively "write" discourse structures. Our approach achieves new state-of-the-art results on the Molweni dataset and overcomes the tree-structure limitations of traditional methods by naturally representing complex graph-based relationships.

Introduction

In natural conversation, utterances are interconnected through logical links such as Elaboration, Acknowledgment, or Question-Answer Pairs. Automatically identifying these links is the goal of Dialogue Discourse Parsing. Historically, state-of-the-art models have relied on encoder-only architectures (e.g., BERT) coupled with task-specific decoding algorithms like Maximum Spanning Tree. While effective, these methods often require significant engineering for different datasets and are typically constrained to predicting tree structures, failing to capture the multi-parent graph structures (Directed Acyclic Graphs) common in real-world dialogue.

Comparison of traditional dialogue discourse parsing vs. the proposed Seq2Seq approach.

Methodology

We propose a generative framework that treats discourse structure as a language. By training an LLM to generate a text-based description of the discourse graph, the model can leverage its pre-trained semantic knowledge to validate logical links.

Seq2Seq-DDP (End-to-End)

In this approach, the model receives the entire dialogue and generates the full structure in a single pass. We experimented with different output formats, including a Natural Scheme (e.g., [e1] is Elaboration of [e0]) and an Augmented Scheme that replicates input text alongside labels to minimize ambiguity.

Seq2Seq-DDP+Transition

To mitigate issues with long-range dependencies and "counting" errors (misidentifying EDU indices), we developed a transition-based system. This model processes one EDU at a time using a sliding window, predicting specific linking actions for the current unit before proceeding. This focused approach significantly reduces hallucinations and improves accuracy in complex dialogues.

Experimental Setup

We evaluated our models on two primary benchmarks:

  1. STAC: A dataset of chat logs from the online game Settlers of Catan, characterized by short, informal utterances.
  2. Molweni: A large-scale dataset derived from the Ubuntu Chat Corpus, focusing on technical support dialogues.

Results and Discussion

Our Seq2Seq-DDP+Transition model established a new state-of-the-art on the Molweni dataset, achieving a Link F1 score of 83.4 and a Full F1 (link + relation) of 60.3. On the STAC dataset, the model remained highly competitive with specialized, heavily engineered baselines.

Performance comparison of Seq2Seq-DDP+Transition against existing state-of-the-art models on STAC and Molweni datasets.

A key advantage of the generative approach is its ability to represent Directed Acyclic Graphs (DAGs). Unlike traditional parsers that force tree outputs, our model can naturally generate multiple incoming links for a single EDU. We observed a 13% precision rate in capturing these complex graph structures, a significant improvement over previous methodologies.

Conclusion

Framing dialogue discourse parsing as a generation task simplifies the modeling pipeline while enhancing performance and structural flexibility. The success of instruction-tuned models like T0-3B suggests that the ability to follow task formats is a critical component of effective parsing. Future research will explore the application of larger decoder-only models and the extension of this generative paradigm to other areas of structural NLP.

Summary updated

BibTeX
@inproceedings{li2024dialogue,
  author = {Chuyuan Li and Yuwei Yin and Giuseppe Carenini},
  title = {{Dialogue Discourse Parsing as Generation: A Sequence-to-Sequence LLM-based Approach}},
  booktitle = {SIGDIAL},
  year = {2024},
  doi = {10.18653/v1/2024.sigdial-1.1},
}
Figure