TA&AT: Enhancing Task-Oriented Dialog with Turn-Level Auxiliary Tasks and Action-Tree Based Scheduled Sampling

Longxiang Liu, Xiuxing Li, Yang Feng

Abstract

Task-oriented dialog systems have witnessed substantial progress due to conversational pre-training techniques. Yet, two significant challenges persist. First, most systems primarily utilize the latest turn's state label for the generator, overlooking the comprehensive value of state labels in boosting the model's understanding for future generations. Second, an overreliance on generated policy often leads to error accumulation, resulting in suboptimal responses when adhering to incorrect actions. To combat these challenges, we propose turn-level multi-task objectives for the encoder. With the guidance of essential information from labeled intermediate states, we establish a more robust representation for both understanding and generation. For the decoder, we introduce an action tree-based scheduled sampling technique. Specifically, we model the hierarchical policy as trees and utilize the similarity between trees to sample negative policy based on scheduled sampling, hoping the model to generate invariant responses under perturbations. This method simulates potential pitfalls by sampling similar negative policy, bridging the gap between task-oriented dialog training and inference. Among methods without continual pre-training, our approach achieved state-of-the-art (SOTA) performance on the MultiWOZ dataset series and was also competitive with pre-trained SOTA methods.

Introduction

Building an end-to-end Task-Oriented Dialog (TOD) system—like a digital assistant that books a restaurant—is a complex pipeline involving Natural Language Understanding (NLU), Dialog State Tracking (DST), Policy Planning (POL), and Natural Language Generation (NLG). Despite recent advances, current systems face two major bottlenecks:

  1. Underutilized Labels: Most models use intermediate labels (like the "belief state" or "system action") only as targets for the generator. They miss the opportunity to use these labels to help the encoder build a deeper semantic understanding of the conversation history.
  2. Exposure Bias & Error Accumulation: During training, models are usually "teacher-forced" with ground-truth actions. At inference, however, the model must rely on its own potentially incorrect generated actions. This mismatch often leads to a "death spiral" where one small mistake in policy leads to a completely nonsensical response.

Methodology

The TA&AT framework addresses these challenges through a two-part solution: TA (Turn-level Auxiliary tasks) and AT (Action-Tree based scheduled sampling).

Overview of the TA&AT framework, illustrating the turn-level auxiliary heads for the encoder and the action-tree based scheduled sampling mechanism for the decoder.

1. Turn-Level Auxiliary Tasks (TA)

Instead of just predicting the next token, the encoder is forced to solve four high-level classification tasks based on the current turn's hidden states:

  • Slot Type Prediction: Which categories (e.g., food, area) are being discussed?
  • Slot Change Prediction: Is the model keeping, changing, deleting, or adding a new constraint?
  • Action Prediction: What is the system's intent (e.g., "inform" or "offer book")?
  • Response Keywords Prediction: Predicting essential "delexicalized" words (like [value_price]) that should appear in the final output.

By supervising the encoder with these turn-level labels, the model develops a "big picture" view of the dialog state, improving its internal representation.

2. Action-Tree Based Scheduled Sampling (AT)

Standard scheduled sampling (randomly replacing a word during training) doesn't work well for dialog because token-level choices are too deterministic. Instead, TA&AT models the system's action sequence as a hierarchical tree (Domain → Action → Slot). During training, the model occasionally replaces the ground-truth action with a "similar" negative sample, with similarity calculated using Tree Edit Distance. This teaches the model to generate a reasonable response even when the preceding policy prediction is slightly off, making the system much more robust during real-world deployment.

Key Results and Insights

The model was evaluated on the industry-standard MultiWOZ 2.0, 2.1, and 2.2 benchmarks, as well as in low-resource settings (10%, 20%, and 50% data).

  • State-of-the-Art Performance: TA&AT achieved the highest "Combined Score" among all methods that do not use expensive continual pre-training. It even remained competitive against massive pre-trained models like SPACE.
  • Robustness: In low-resource experiments, TA&AT significantly outperformed baselines, showing that auxiliary tasks help the model learn more effectively from fewer examples.
  • The Power of Perturbation: By intentionally feeding the model "near-miss" incorrect actions during training, the decoder learns to rely more on the conversation context rather than blindly following a potentially flawed policy.
Qualitative comparison of responses: TA&AT provides comprehensive information matching the ground truth, while the Mars baseline fails to provide the phone number and address.

Limitations and Future Work

While TA&AT shows significant improvements, the authors noted challenges in Task Balancing. Different auxiliary tasks have different learning curves; for instance, the Response Keywords task sometimes underperformed because it was still being optimized while other tasks had already converged. Future research will explore more sophisticated ways to balance multi-task learning objectives to prevent one task from dominating gradient updates.

Summary updated

Abstract

Task-oriented dialog systems have witnessed substantial progress due to conversational pre-training techniques. Yet, two significant challenges persist. First, most systems primarily utilize the latest turn's state label for the generator. This practice overlooks the comprehensive value of state labels in boosting the model's understanding for future generations. Second, an overreliance on generated policy often leads to error accumulation, resulting in suboptimal responses when adhering to incorrect actions. To combat these challenges, we propose turn-level multi-task objectives for the encoder. With the guidance of essential information from labeled intermediate states, we establish a more robust representation for both understanding and generation. For the decoder, we introduce an action tree-based scheduled sampling technique. Specifically, we model the hierarchical policy as trees and utilize the similarity between trees to sample negative policy based on scheduled sampling, hoping the model to generate invariant responses under perturbations. This method simulates potential pitfalls by sampling similar negative policy, bridging the gap between task-oriented dialog training and inference. Among methods without continual pre-training, our approach achieved state-of-the-art (SOTA) performance on the MultiWOZ dataset series and was also competitive with pre-trained SOTA methods.
BibTeX
@inproceedings{liu2024ta,
  author = {Longxiang Liu and Xiuxing Li and Yang Feng},
  title = {{TA\&AT: Enhancing Task-Oriented Dialog with Turn-Level Auxiliary Tasks and Action-Tree Based Scheduled Sampling}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2024},
  url = {https://openreview.net/forum?id=KitRgxAH9M},
}
Figure