Abstract
Existing dialog state tracking (DST) models are trained with dialog data in a random order, neglecting rich structural information in a dataset. In this paper, we propose to use curriculum learning (CL) to better leverage both the curriculum structure and schema structure for task-oriented dialogs. Specifically, we propose a model-agnostic framework called Schema-aware Curriculum Learning for Dialog State Tracking (SaCLog), which consists of a preview module that pre-trains a DST model with schema information, a curriculum module that optimizes the model with CL, and a review module that augments mispredicted data to reinforce the CL training. We show that our proposed approach improves DST performance over both a transformer-based and RNN-based DST model (TripPy and TRADE) and achieves new state-of-the-art results on WOZ2.0 and MultiWOZ2.1.
Introduction
Dialog State Tracking (DST) is a critical component of task-oriented dialog systems, responsible for maintaining an internal representation of the user's goals and constraints throughout a conversation. This state is typically represented as a set of slot-value pairs (e.g., food=italian, area=centre). Despite significant progress, current DST models often struggle with two key aspects: the inherent difficulty variation across different dialogs and the underutilization of the underlying domain schema.
Most training regimes treat all dialog samples as equally informative and present them in a random order. However, a single-turn request for a taxi is fundamentally "easier" than a multi-turn negotiation involving restaurant bookings and hotel reservations with complex coreferences. Furthermore, while schemas define the relationships between domains and slots, they are rarely used to guide the learning process or generate high-quality synthetic data for hard-to-learn cases.
The SaCLog Framework
To address these challenges, we introduce SaCLog (Schema-aware Curriculum Learning), a model-agnostic framework designed to mimic the human learning process: previewing the material, attending to lessons in increasing order of difficulty, and reviewing mistakes.
1. Preview Module (Pre-training)
The Preview module focuses on schema-aware pre-training. It uses a slot encoder and a dialog context encoder to learn the relationships between specific slots and conversation tokens. The model is trained on two objectives:
- Token Prediction: Identifying which tokens in the context correspond to a given slot.
- Status Change Detection: Determining if a slot's value has been added, deleted, or changed in the current turn. This phase provides the model with a strong inductive bias regarding the schema before it encounters real-world dialog data.
2. Curriculum Module (Attend)
The "Attend" phase implements curriculum learning using a "baby step" strategy. Dialogs are sorted into "buckets" based on a hybrid difficulty score:
- Model-based Difficulty: Calculated using the uncertainty or error rate of a committee of models.
- Rule-based Difficulty: Derived from structural features such as turn count, token count, and the number of entities mentioned. The training starts with the easiest bucket. The model only progresses to more difficult data once it has converged on the current set, ensuring a stable and efficient learning trajectory.
3. Review Module (Data Augmentation)
In the "Review" phase, the model identifies the top 10% of examples it mispredicted during the curriculum phase. It then uses a Schema-based Augmenter to generate new training samples through three techniques:
- Slot Substitution: Swapping related slots (e.g., changing "destination" to "departure").
- Value Replacement: Changing specific values (e.g., "cheap" to "mid-priced").
- Dialog Recombination: Mixing turns from different dialogs to create new complex scenarios. This targeted augmentation reinforces the model’s understanding of its specific weaknesses.
Experimental Evaluation
We evaluated SaCLog on two major benchmarks: MultiWOZ 2.1 (multi-domain) and WOZ 2.0 (single-domain). We applied the framework to two distinct base architectures: the transformer-based TripPy and the RNN-based TRADE.
As shown in the results table, TripPy + SaCLog achieves a new state-of-the-art Joint Goal Accuracy (JGA) of 60.61% on MultiWOZ 2.1, significantly outperforming the base TripPy model and other competitive methods like CoCoAug. On the WOZ 2.0 dataset, it reaches 94.2% JGA. The consistent improvements across both TRADE and TripPy demonstrate that SaCLog is indeed model-agnostic and robust.
Conclusion
SaCLog demonstrates that structured learning, inspired by human pedagogy, can significantly enhance the performance of dialog state trackers. By integrating schema information into pre-training, employing a hybrid curriculum, and using targeted review via data augmentation, the framework achieves state-of-the-art results while remaining compatible with various underlying model architectures. Future work will explore the application of this "Preview, Attend, Review" paradigm to other dialog tasks such as policy learning and response generation.
Reference: Yinpei Dai, Hangyu Li, Yongbin Li, Jian Sun, Fei Huang, Luo Si, Xiaodan Zhu. (2021). Preview, Attend and Review: Schema-Aware Curriculum Learning for Multi-Domain Dialog State Tracking. arXiv:2106.00291v1 [cs.CL].
Summary updated