Exploring Auxiliary Reasoning Tasks for Task-oriented Dialog Systems with Meta Cooperative Learning

Bowen Qin, Min Yang, Lidong Bing, Qingshan Jiang, Chengming Li, Ruifeng Xu

Abstract

Task-oriented dialog systems (TDSs) face the dual challenge of maintaining factual accuracy through Knowledge Base (KB) retrieval while ensuring linguistic coherence in conversation. Standard end-to-end models often struggle to balance these requirements, frequently requiring expensive, handcrafted labels for auxiliary tasks. This paper introduces Meta Cooperative Learning (MCL), a framework that trains a primary TDS alongside two specialized "tutor" tasks: KB reasoning and dialog reasoning. By leveraging a meta-learning strategy to automatically generate auxiliary labels, MCL creates a feedback loop that enhances both entity retrieval and natural language generation. Evaluations on CamRest, In-Car Assistant, and Multi-WOZ 2.1 benchmarks show that MCL significantly outperforms state-of-the-art baselines, particularly in maintaining stability across multi-turn dialogues.

Introduction

Task-oriented dialog systems are essential for applications such as restaurant reservations or navigation assistance. However, they must be both factually accurate (retrieving correct entities from a KB) and linguistically coherent. Existing models often fail to balance these aspects, leading to "hallucinated" facts or ungrammatical responses. Furthermore, scaling these systems usually involves manual labeling for auxiliary tasks like dialog state tracking, which is labor-intensive. MCL addresses these issues by disentangling reasoning tasks and using meta-learning to automate the training of auxiliary components.

Methods: Meta Cooperative Learning (MCL)

The MCL framework consists of three interconnected networks designed to isolate and then recombine different types of reasoning:

  1. Primary Task: A standard TDS that generates the final response using all available information.
  2. Auxiliary KB Reasoning Task: A model variant stripped of dialog memory, forcing a focus on extracting correct entities from the KB.
  3. Auxiliary Dialog Reasoning Task: A variant stripped of KB access, focusing purely on language patterns and conversational flow.
The Meta Cooperative Learning framework, illustrating the interaction between the primary task and the two auxiliary reasoning tasks.

The core innovation is the "Meta" feedback loop. Instead of human labels, MCL uses a two-stage gradient strategy. In the first stage, the primary network is trained using advice (auxiliary labels) from the specialized networks. In the second stage, the auxiliary networks are updated based on how effectively their advice improved the primary network's performance. This allows the "tutors" to learn how to better guide the "student" without manual intervention.

Experimental Setup

The model was evaluated on three major benchmarks:

  • CamRest: 676 multi-turn dialogs for restaurant reservations.
  • In-Car Assistant: 3,031 dialogs covering scheduling, weather, and navigation.
  • Multi-WOZ 2.1: A large-scale dataset with 1,839 dialogs across multiple domains.

Results and Discussion

MCL outperformed strong baselines like GLMP and MLM across all datasets. On Multi-WOZ 2.1, MCL achieved a BLEU score of 13.6, a 48% improvement over the MLM baseline (9.2). In terms of factual accuracy, MCL reached an Entity F1 score of 59.2 on CamRest, surpassing GLMP's 50.6.

Average BLEU score vs. number of turns, showing MCL's superior stability in long-turn dialogues compared to baselines.

A key finding is MCL's stability over time. As shown in the figure above, while baseline models like Seq2Seq experience a significant drop in BLEU scores after 3-4 turns, MCL maintains high-quality output even in deep dialogues. Human evaluations also confirmed that MCL produces more informative and human-like responses.

Conclusion

Meta Cooperative Learning provides a robust framework for balancing the competing demands of task-oriented dialog. By disentangling KB and dialog reasoning and automating the learning process through meta-learning, the system achieves state-of-the-art performance without the need for extensive manual labeling. Future work may address the computational overhead of the meta-learning process and explore the application of this framework to unstructured knowledge sources.

Summary updated

Abstract

In this paper, we propose a Meta Cooperative Learning (MCL) framework for task-oriented dialog systems (TDSs). Our model consists of an auxiliary KB reasoning task for learning meta KB knowledge, an auxiliary dialogue reasoning task for learning dialogue patterns, and a TDS task (primary task) that aims at not only retrieving accurate entities from KB but also generating natural responses, which are coordinated to achieve collective success in both retrieving accurate KB entities and generating human-like responses via meta learning. Concretely, the dialog generation model amalgamates complementary meta KB and dialog knowledge from two novel auxiliary reasoning tasks that together provide integrated guidance to build a high-quality TDS by adding regularization terms to force primary network to produce similar results to auxiliary networks. While MCL automatically learns appropriate labels for the two auxiliary reasoning tasks from the primary task, without requiring access to any further data. The key idea behind MCL is to use the performance of the primary task, which is trained alongside the auxiliary tasks in one iteration, to improve the auxiliary labels for the next iteration with meta learning. Experimental results on three benchmark datasets show that MCL can generate higher quality responses compared to several strong baselines in terms of both automatic and human evaluations. Code to reproduce the results in this paper is available at: https://github.com/siat-nlp/MCL.
BibTeX
@inproceedings{qin2021exploring,
  author = {Bowen Qin and Min Yang and Lidong Bing and Qingshan Jiang and Chengming Li and Ruifeng Xu},
  title = {{Exploring Auxiliary Reasoning Tasks for Task-oriented Dialog Systems with Meta Cooperative Learning}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2021},
  doi = {10.1609/aaai.v35i15.17615},
}
Figure