Mars: Semantic-aware Contrastive Learning for End-to-End Task-Oriented Dialog

Haipeng Sun, Junwei Bao, Youzheng Wu, Xiaodong He

Abstract

Traditional end-to-end task-oriented dialog systems first convert dialog context into belief state and action state before generating the system response. The system response performance is significantly affected by the quality of the belief state and action state. We first explore what dialog context representation is beneficial to improving the quality of the belief state and action state, which further enhances the generated response quality. To tackle our exploration, we propose Mars, an end-to-end task-oriented dialog system with two contrastive learning strategies to model the relationship between dialog context and belief/action state representations. Empirical results show dialog context representations, which are more different from semantic state representations, are more conducive to multi-turn task-oriented dialog. Moreover, our proposed Mars achieves state-of-the-art performance on the MultiWOZ 2.0, CamRest676, and CrossWOZ.

Introduction

Task-oriented dialog (TOD) systems, such as those used for booking travel or managing reservations, typically follow a pipeline: they process dialog context (history), extract a belief state (user constraints), determine an action state (system strategy), and generate a response. In modern end-to-end models, these steps are integrated into a single neural architecture. However, the performance of these systems is often bottlenecked by the quality of the intermediate semantic states.

Mars is a novel end-to-end TOD system designed to bridge the gap between raw conversation history and structured semantic states. The core research question addressed is how to represent the relationship between messy natural language context and structured logic to maximize performance.

Methodology

Mars utilizes a shared encoder and two specialized decoders (one for belief states and one for action/response generation). The primary innovation is the introduction of two contrastive learning strategies to refine the internal representations:

  1. Pair-aware Context & State (Mars-P): This strategy focuses on semantic consistency. It uses a contrastive loss to pull the vector representation of a specific dialog turn closer to its corresponding ground-truth state, while pushing it away from unrelated states.
  2. Group-aware Context & State (Mars-G): This approach forces the model to enlarge the margin between all context representations and all state representations. This prevents information redundancy, compelling the model to learn richer, more diverse features for the context rather than simply mimicking the output state.
Overview of the Mars-P and Mars-G contrastive learning strategies.

Experimental Setup

The model was evaluated on three diverse benchmarks to test its robustness and cross-lingual capabilities:

  • MultiWOZ 2.0: A large-scale, multi-domain English TOD dataset.
  • CamRest676: A specialized English dataset for restaurant reservations.
  • CrossWOZ: A large-scale, multi-domain Chinese TOD dataset.

Key Insights and Results

The most significant finding of the study is that "different is better." Mars-G, which emphasizes the distinction between context and state, consistently outperformed Mars-P, which emphasizes similarity. This suggests that maintaining unique information in the context representation is crucial for complex, multi-turn reasoning.

Visualization of dialog context and state representations across different models, showing the separation achieved by Mars-G.

Performance Highlights

  • State-of-the-Art Results: Mars achieved SOTA performance across all datasets. On MultiWOZ 2.0, it reached a Combined Score of 103.4, surpassing previous benchmarks like GALAXY and MTTOD.
  • Low-Resource Efficiency: Mars-G is remarkably effective with limited data. When trained on just 20% of the available data, it outperformed several established models trained on the full 100%.
  • Long-Range Dependencies: The contrastive approach proved particularly beneficial as conversations grew longer, widening the performance gap over baseline models in multi-turn scenarios.

Conclusion and Future Work

While Mars significantly advances the state of end-to-end TOD systems, it still faces challenges. The training process is supervision-heavy, requiring expensive human-annotated states. Furthermore, error analysis indicates that 64% of failures still stem from inaccurate state predictions. Future research will focus on semi-supervised methods to reduce data dependency and further improve state accuracy for real-world industrial deployment.

Summary updated

Abstract

Traditional end-to-end task-oriented dialog systems first convert dialog context into belief state and action state before generating the system response. The system response performance is significantly affected by the quality of the belief state and action state. We first explore what dialog context representation is beneficial to improving the quality of the belief state and action state, which further enhances the generated response quality. To tackle our exploration, we propose Mars, an end-to-end task-oriented dialog system with two contrastive learning strategies to model the relationship between dialog context and belief/action state representations. Empirical results show dialog context representations, which are more different from semantic state representations, are more conducive to multi-turn task-oriented dialog. Moreover, our proposed Mars achieves state-of-the-art performance on the MultiWOZ 2.0, CamRest676, and CrossWOZ.
BibTeX
@inproceedings{sun2023mars,
  author = {Haipeng Sun and Junwei Bao and Youzheng Wu and Xiaodong He},
  title = {{Mars: Semantic-aware Contrastive Learning for End-to-End Task-Oriented Dialog}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2023},
  url = {https://openreview.net/forum?id=2-XVYllg3v},
}
Figure