A Practical Dialogue-Act-Driven Conversation Model for Multi-Turn Response Selection

Harshit Kumar, Arvind Agarwal, Sachindra JoshiEMNLP 2019 · DOI

Abstract

Selecting the most appropriate response in a multi-turn conversation requires an understanding of both semantic content and communicative intent. While Dialogue Acts (DAs) provide crucial structural information, their unavailability at test time and the risk of error propagation from automated classifiers have limited their practical application. This paper introduces a multi-task learning framework that predicts DAs on the fly and integrates them with utterance text through a novel "Crossway" architecture. Our model, Siamese-PDA-MT+Crossway, achieves performance levels nearly identical to systems using ground-truth human labels, offering a robust and practical solution for real-world conversational agents.

Introduction

In conversational AI, response selection involves identifying the best reply from a candidate pool based on the dialogue history. Dialogue Acts (DAs)—labels such as "Question," "Statement," or "Agreement"—are known to be highly predictive of appropriate responses. However, two primary challenges persist: the unavailability of human-labeled DAs in real-time applications and the "poisoning" effect of errors when DAs are predicted by an external model. This research addresses these gaps by developing a model that internalizes DA prediction and remains resilient to its own classification noise.

Methodology

The proposed architecture, Siamese-PDA-MT+Crossway, utilizes a multi-task learning approach to jointly optimize for dialogue act classification and response ranking.

Shared Encoder and Multi-Task Learning

The model employs a single Gated Recurrent Unit (GRU) to encode both the conversation context and candidate responses. This shared representation ensures that the learned features are optimized for both identifying the communicative "act" and selecting the most relevant "response." The training objective is two-fold:

  1. DA Prediction: Classifying the dialogue act for each utterance in the context and the candidate response.
  2. Response Selection: Ranking candidate responses based on their similarity to the context.

The Crossway Module

The core innovation is the "Crossway" module, which synthesizes four distinct representations: context text, response text, predicted context DA, and predicted response DA. By concatenating text representations with their corresponding predicted DA vectors, the model creates "compositional" representations. These are then compared using a bilinear function to produce a final similarity score.

Architecture of the Siamese-PDA-MT+Crossway model showing the multi-task flow and Crossway integration.

Experimental Setup

The model was evaluated on two benchmark datasets:

  • DailyDialog: Human-to-human conversations annotated with four broad DA classes (Inform, Question, Directive, Commissive).
  • Switchboard Dialogue Act Corpus (SwDA): Telephonic conversations featuring 42 fine-grained DA classes.

Results and Analysis

The multi-task Crossway model consistently outperformed standard Siamese networks and single-task variants across both datasets.

Performance comparison of the proposed model against baselines on DailyDialog and SwDA.

Performance Gains and Closing the Gap

On the DailyDialog dataset, the model achieved a Mean Reciprocal Rank (MRR) of 0.946, a substantial improvement over the baseline Siamese model (0.867). Notably, this performance is remarkably close to the "Upper Bound" model (MRR 0.956), which utilizes perfect human-labeled DAs. This suggests that the Crossway architecture effectively mitigates the impact of internal prediction errors.

Key Insights

  • Joint Training Synergy: Training for DA prediction acts as a regularizer, forcing the encoder to capture deeper semantic structures. This synergy improved DA prediction accuracy itself (e.g., from 82.9% to 86.1% on DailyDialog).
  • Importance of Response DAs: While previous work often focused solely on context DAs, this study highlights that predicting the DA of the candidate response is equally critical for accurate matching.

Conclusion

The Siamese-PDA-MT+Crossway model demonstrates that Dialogue Acts can be effectively leveraged in a practical, end-to-end system without requiring external labels at inference time. By integrating DA prediction as a parallel task and using a compositional architecture, the model achieves state-of-the-art performance while remaining robust to prediction noise. Future research will explore scaling this approach to generative models and longer conversation histories.

Summary updated

BibTeX
@inproceedings{kumar2019practical,
  author = {Harshit Kumar and Arvind Agarwal and Sachindra Joshi},
  title = {{A Practical Dialogue-Act-Driven Conversation Model for Multi-Turn Response Selection}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2019},
  doi = {10.18653/v1/d19-1205},
}
Figure