On Tracking Dialogue State by Inheriting Slot Values in Mentioned Slot Pools

Zhoujian Sun, Zhengxing Huang, Nai Ding

Abstract

Dialogue state tracking (DST) is a critical component of task-oriented dialogue systems, responsible for extracting and managing slot values from user utterances. Traditional models often suffer from error propagation due to "naïve inheritance" of previous states and struggle with indirect mentions. This study introduces a model utilizing a Mentioned Slot Pool (MSP)—a slot-specific memory that records all mentioned values. By allowing the model to reject previous values and re-extract them from updated context, the MSP approach significantly reduces inheritance errors. Experimental results demonstrate that the MSP model achieves state-of-the-art performance on the MultiWOZ 2.1 and 2.2 datasets while maintaining high parameter efficiency.

Introduction

In task-oriented dialogue systems, such as those used for booking travel or restaurants, Dialogue State Tracking (DST) serves as the engine that manages "slots" (e.g., destination, time, number of people). A persistent challenge in DST is the management of slot values across multiple turns. Most existing models rely on a simple inheritance mechanism: if a slot is not explicitly mentioned in the current turn, the model carries over the value from the previous turn.

This approach leads to two primary issues:

  1. Error Propagation: An early misinterpretation (e.g., misidentifying a date) is inherited turn after turn, leading to total task failure.
  2. Indirect Mentions: Users frequently refer to previous information implicitly (e.g., "the same day as my train"). Standard models often fail to map these cross-slot dependencies correctly.

Methods: The Mentioned Slot Pool (MSP)

The proposed architecture replaces static inheritance with a dynamic Mentioned Slot Pool (MSP). The MSP acts as a slot-specific memory, storing all values mentioned throughout the conversation.

The MSP model architecture, highlighting the Hit Type Prediction and the Mentioned Slot Pool (MSP) for value selection.

The model processes each turn through a three-step pipeline:

  • Hit Type Prediction: The model classifies each slot into one of four categories: none, dontcare, mentioned (value exists in the MSP), or hit (value must be extracted from the current text). If a hit is predicted for an already-filled slot, the model effectively "rejects" the previous state and re-analyzes the context.
  • Mentioned Value Selection: For slots categorized as mentioned, a bilinear model selects the most probable value from the MSP, facilitating the resolution of indirect references.
  • Hit Value Prediction: New values are extracted using either a classifier (for categorical slots) or a span-predictor (for open-ended text) based on the current dialogue context.

The system utilizes BERT (base and large) as the underlying language model to generate contextual embeddings.

Results and Discussion

The MSP model was evaluated on the MultiWOZ 2.1, MultiWOZ 2.2, and WOZ 2.0 benchmarks. As shown in the performance comparison, the MSP model achieved new state-of-the-art (SOTA) results among models not utilizing external data augmentation.

Comparison of the MSP model against other state-of-the-art DST models on MultiWOZ and WOZ datasets.

Key findings include:

  • Performance: The MSP-L model reached 57.2% Joint Goal Accuracy (JGA) on MultiWOZ 2.1 and 57.7% on MultiWOZ 2.2.
  • Efficiency: The MSP-B model (110M parameters) outperformed significantly larger models like AG-DST-A (680M parameters), demonstrating that intelligent memory management is more effective than sheer model scale.
  • Error Reduction: Ablation studies confirmed that the MSP module is responsible for a 2-3% performance gain, primarily by reducing "Inherit Errors"—instances where the model incorrectly carries over a previous mistake.

Conclusion

By treating dialogue state as a dynamic memory rather than a static inheritance, the MSP model effectively breaks the cycle of error propagation in DST. While the model shows significant improvement, it still faces challenges with complex semantic reasoning, such as handling mid-sentence user corrections. Future research will focus on integrating external linguistic knowledge to address these deep semantic hurdles and further refine the "re-analysis" strategy.

Summary updated

Abstract

Dialogue state tracking (DST) is a component of the task-oriented dialogue system. It is responsible for extracting and managing slot values according to dialogue utterances, where each slot represents an essential part of the information to accomplish a task, and slot value is updated recurrently in each dialogue turn. However, many DST models cannot update slot values appropriately. These models may repeatedly inherit wrong slot values extracted in previous turns, resulting in the fail of the entire DST task. They cannot update indirectly mentioned slots well, either. This study designed a model with a mentioned slot pool (MSP) to tackle the update problem. The MSP is a slot-specific memory that records all mentioned slot values that may be inherited, and our model updates slot values according to the MSP and the dialogue context. Our model rejects inheriting the previous slot value when it predicates the value is wrong. Then, it re-extracts the slot value from the current dialogue context. As the contextual information accumulates with the dialogue progress, the new value is more likely to be correct. It also can track the indirectly mentioned slot by picking a value from the MSP. Experimental results showed our model reached state-of-the-art DST performance on MultiWOZ 2.1 and 2.2 datasets.
BibTeX
@inproceedings{sun2022tracking,
  author = {Zhoujian Sun and Zhengxing Huang and Nai Ding},
  title = {{On Tracking Dialogue State by Inheriting Slot Values in Mentioned Slot Pools}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2022},
  url = {https://openreview.net/forum?id=-TnimbQ4tFg},
}
Figure