UniDS: A Unified Dialogue System for Chit-Chat and Task-oriented Dialogues

Xinyan Zhao, Bin He, Yasheng Wang, Yitong Li, Fei Mi, Yajiao Liu, Xin Jiang, Qun Liu, Huanhuan Chen

TL;DR

This paper introduces UniDS, a novel unified dialogue system capable of seamlessly handling both chit-chat and task-oriented conversations. By designing a unified data schema and training on mixed dialogue data, UniDS achieves comparable chit-chat performance while outperforming existing task-oriented systems, all without increasing model complexity.

What Problem Does This Paper Address?

Current dialogue systems are largely specialized, with separate models for engaging in casual conversation (chit-chat) and completing specific tasks (task-oriented dialogue, TOD). This separation limits the naturalness of human-AI interaction, as users often switch between conversational modes within a single interaction. For instance, a user might inquire about a hotel's price and then ask about the weather. Building a single agent that can fluidly navigate these different conversational styles is a significant challenge. Furthermore, real-world dialogue systems must be robust to noisy inputs and the ability to gracefully handle unexpected conversational shifts.

Methods

The core innovation of UniDS lies in its unified approach. The authors propose a unified dialogue data schema that extends task-oriented dialogue components like "belief state" and "system act" to chit-chat dialogues. For chit-chat, belief states capture key nouns from user utterances, and system acts are marked with a special "[chit_chat]" token. This unified schema allows for training a single model on a mixed dataset of both chit-chat and TOD conversations.

UniDS is built upon a pre-trained language model (specifically, it leverages a pre-trained chit-chat model like DialoGPT) and is trained in an end-to-end fashion. The training process involves generating the belief state, database results, system act, and finally the response, all autoregressively. To address the differing objectives of chit-chat (engagement) and TOD (task completion), they introduce a weighted cross-entropy loss that gives more importance to "entity recommendation" acts, crucial for task success.

Datasets

The authors train and evaluate UniDS on two main datasets:

  • Chit-chat Dataset: An 8k dialogue dataset extracted from Reddit dumps, filtered for quality and conversation length.
  • MultiWOZ: A widely used, large-scale, multi-domain task-oriented dialogue dataset.

The UniDS system is trained on a mixture of these datasets, processed according to the proposed unified data schema.

Key Insights and Ideas

  • Unified Schema is Key: The success of UniDS hinges on the novel unified data schema, which bridges the gap between chit-chat and TOD by incorporating belief states and system acts into chit-chat dialogues. This allows a single model to learn from both types of data.
  • Pre-trained Chit-Chat as a Foundation: Leveraging a pre-trained chit-chat model as a starting point proves effective, allowing UniDS to retain strong conversational abilities while learning task-oriented skills.
  • Weighted Loss for Task Focus: The weighted cross-entropy loss is crucial for balancing chit-chat and TOD objectives, specifically by prioritizing entity recommendation acts, which directly contribute to task completion.
  • Switching is Natural and Achievable: The paper demonstrates that a unified system can indeed switch between dialogue types, a critical capability for real-world applications.

Key Results

  • Comparable Chit-Chat Performance: UniDS achieves performance comparable to state-of-the-art chit-chat systems like DialoGPT, even after being trained on mixed data.
  • Outperforms TOD Systems: UniDS outperforms existing task-oriented dialogue systems (like UBAR) in terms of Inform, Success, and Combined scores, suggesting that chit-chat training enhances TOD capabilities.
  • Robustness to Noise: UniDS shows improved robustness to irrelevant chit-chat utterances inserted into task-oriented dialogues compared to UBAR.
  • Effective Switching: UniDS demonstrates a high success rate (around 98%) in switching between chit-chat and TOD within two turns, showcasing its ability to adapt to user conversational shifts.

Limitations and Future Work

  • Switching Transition: The paper notes that when switching from TOD to chit-chat, the model's "Switch-1" metric is relatively low, suggesting a tendency to provide transitional responses rather than immediately shifting to a chatty mode.
  • Parameter Efficiency: While UniDS avoids adding extra parameters to SOTA baselines, the paper doesn't explicitly detail the computational overhead compared to separate models.
  • Future Work: The authors suggest that their work inspires future studies towards building more capable and intelligent dialogue systems. Further exploration into optimizing the switching mechanism and exploring more complex multi-domain scenarios could be valuable.

Summary updated

BibTeX
@inproceedings{zhao2022unids,
  author = {Xinyan Zhao and Bin He and Yasheng Wang and Yitong Li and Fei Mi and Yajiao Liu and Xin Jiang and Qun Liu and Huanhuan Chen},
  title = {{UniDS: A Unified Dialogue System for Chit-Chat and Task-oriented Dialogues}},
  booktitle = {DialDoc @ ACL},
  year = {2022},
  doi = {10.18653/v1/2022.dialdoc-1.2},
}
Figure