Improving Dialogue Discourse Parsing via Reply-to Structures of Addressee Recognition

Yaxin Fan, Feng Jiang, Peifeng Li, Fang Kong, Qiaoming ZhuEMNLP 2023 · OpenReview

Abstract

Dialogue discourse parsing aims to reflect the relation-based structure of dialogue by establishing discourse links according to discourse relations. To alleviate data sparsity, previous studies have adopted multitasking approaches to jointly learn dialogue discourse parsing with related tasks (e.g., reading comprehension) that require additional human annotation, thus limiting their generality. In this paper, we propose a multitasking framework that integrates dialogue discourse parsing with its neighboring task, addressee recognition. Addressee recognition reveals the reply-to structure that partially overlaps with the relation-based structure, which can be exploited to facilitate relation-based structure learning. To this end, we first proposed a reinforcement learning agent to identify training examples from addressee recognition that are most helpful for dialogue discourse parsing. Then, a task-aware structure transformer is designed to capture the shared and private dialogue structure of different tasks, thereby further promoting dialogue discourse parsing. Experimental results on both the Molweni and STAC datasets show that our proposed method can outperform the SOTA baselines.

The Challenge of Dialogue Discourse Parsing

Dialogue Discourse Parsing (DDP) is the task of mapping out the structural and logical connections in a conversation—for example, identifying that one utterance is an "Acknowledgement" of another, or that two utterances form a "Question-Answer Pair." This structural understanding is crucial for building AI that can summarize meetings or follow complex multi-party chats.

However, DDP faces a major hurdle: data sparsity. Annotating these logical links is incredibly time-consuming and requires expert linguistic knowledge. Previous attempts to solve this used "neighboring" tasks like reading comprehension to help the model learn, but those tasks also require expensive human labels.

This research focuses on Addressee Recognition (AR)—identifying the "reply-to" structure of a chat. While the "who-replied-to-whom" structure isn't identical to the "logical-relation" structure, they overlap significantly, as shown in Figure 1. The authors aim to leverage the massive amounts of available AR data to boost DDP performance without needing extra manual annotations.

Comparison between reply-to structures (orange) and relation-based discourse structures (black) in a multi-party dialogue.

Proposed Methodology

The authors propose a two-step multitasking framework designed to filter and integrate addressee recognition data effectively.

The proposed two-step framework: (Step 1) RL-based data selection and (Step 2) Multitask Learning using a Task-aware Structure Transformer.

1. Reinforcement Learning (RL) Data Selection

Not all "reply-to" examples are helpful for understanding logical flow. The authors built an Actor-Critic RL agent that acts as a gatekeeper. It evaluates batches of addressee recognition data and decides whether they are "helpful" or "noisy" based on how much they improve the model's performance on a small validation set of discourse parsing. This ensures the model learns from high-quality structural skeletons rather than simple conversational noise.

2. Task-aware Structure Transformer (TST)

Standard multitask learning often suffers from "negative transfer," where learning one task confuses the model on another. The TST addresses this by categorizing individual neurons in the transformer as either task-shared or task-private.

  • Shared neurons are updated by both tasks to capture general conversational patterns.
  • Private neurons are only updated by their specific task, allowing the model to capture the unique nuances of discourse relations versus simple reply-to links.

Experimental Results

The model was evaluated on two primary DDP benchmarks: STAC (online game dialogues) and Molweni (Ubuntu chat logs). For the auxiliary AR task, they utilized approximately 1.75 million dialogues from massive public datasets.

Key Insights

  • Less is More: The RL agent discovered that it could discard about 80% of the addressee recognition data and actually achieve higher performance on discourse parsing. This suggests that much raw conversational data is too structurally simple to help teach complex logic.
  • State-of-the-Art Performance: The proposed model, TST$_{Subset}$, outperformed all previous systems. On Molweni, it achieved a Link F1 score of 85.26 and a Link&Rel F1 of 60.91. On STAC, it reached a Link F1 of 73.69.
  • Efficiency: The model proved that "who-to-whom" interaction data is a powerful, "free" signal that can replace more expensive human-annotated auxiliary tasks.

Conclusion and Future Work

By leveraging the inherent "reply-to" structure of conversations, this framework significantly improves the accuracy of dialogue discourse parsing while reducing reliance on expert annotations. While the improvement was most pronounced in link prediction, future work aims to incorporate multiple different "neighbor" tasks simultaneously to further bridge the gap in low-resource discourse parsing and improve specific relation classification.

Summary updated

Abstract

Dialogue discourse parsing aims to reflect the relation-based structure of dialogue by establishing discourse links according to discourse relations. To alleviate data sparsity, previous studies have adopted multitasking approaches to jointly learn dialogue discourse parsing with related tasks (e.g., reading comprehension) that require additional human annotation, thus limiting their generality. In this paper, we propose a multitasking framework that integrates dialogue discourse parsing with its neighboring task addressee recognition. Addressee recognition reveals the reply-to structure that partially overlaps with the relation-based structure, which can be exploited to facilitate relation-based structure learning. To this end, we first proposed a reinforcement learning agent to identify training examples from addressee recognition that are most helpful for dialog discourse parsing. Then, a task-aware structure transformer is designed to capture the shared and private dialogue structure of different tasks, thereby further promoting dialogue discourse parsing. Experimental results on both the Molweni and STAC datasets show that our proposed method can outperform the SOTA baselines. The code will be available at https://github.com/yxfanSuda/RLTST.
BibTeX
@inproceedings{fan2023improving,
  author = {Yaxin Fan and Feng Jiang and Peifeng Li and Fang Kong and Qiaoming Zhu},
  title = {{Improving Dialogue Discourse Parsing via Reply-to Structures of Addressee Recognition}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2023},
  url = {https://openreview.net/forum?id=7yBFS9GmwS},
}
Figure