On Tracking Dialogue State by Inheriting Slot Values in Mentioned Slot Pools

Zhoujian Sun, Zhengxing Huang, Nai Ding

Abstract

Dialogue state tracking (DST) is a component of the task-oriented dialogue system. It is responsible for extracting and managing slots, where each slot represents a part of the information to accomplish a task, and slot value is updated recurrently in each dialogue turn. However, many DST models cannot update slot values appropriately. These models may repeatedly inherit wrong slot values extracted in previous turns, resulting in the failure of the entire DST task. They cannot update indirectly mentioned slots well, either. This study designed a model with a mentioned slot pool (MSP) to tackle the update problem. The MSP is a slot-specific memory that records all mentioned slot values that may be inherited, and our model updates slot values according to the MSP and the dialogue context. Our model rejects inheriting the previous slot value when it predicts the value is wrong. Then, it extracts the slot value from the current dialogue context. As the contextual information accumulates, the new value is more likely to be correct. It also can track the indirectly mentioned slot by picking a value from the MSP. Experimental results showed our model reached state-of-the-art DST performance on MultiWOZ datasets.

Introduction

Dialogue State Tracking (DST) serves as the core "brain" of task-oriented chatbots, tasked with extracting specific information—slots like destination, time, or price range—from a conversation to fulfill user requests. Despite advancements, current models often fall into the "Inheritance Trap," where an error made in an early turn is blindly carried through the rest of the dialogue. Furthermore, models frequently struggle with indirect mentions, such as when a user says, "Book a restaurant for the same day as my train," failing to link the train-day slot to the restaurant-day slot.

Example of dialogue state tracking across multiple turns, illustrating the complexity of updating slots and handling indirect mentions like 'same day'.

Methodology: The Mentioned Slot Pool (MSP)

To address these limitations, we introduce the Mentioned Slot Pool (MSP), a slot-specific memory mechanism that records all previously mentioned values within a dialogue. This architecture, built upon a Pre-trained Neural Network Language Model (PNNLM) such as BERT, employs a multi-step prediction process for every slot at each turn.

Prediction and Correction Mechanisms

The model classifies each slot into one of four categories via Hit Type Prediction: none, dontcare, mentioned (value exists in the pool), or hit (value is in the current text).

The MSP architecture integrated with a Pretrained Neural Network Language Model (PNNLM), showing the Mentioned Slot Pool and prediction heads.

A critical innovation is the "Reject" Mechanism. If the model predicts a hit for a slot that already possesses a value, it effectively rejects the legacy value and re-extracts it from the current context. This allows the system to correct past mistakes as more information becomes available. For mentioned statuses, the model uses a bilinear attention mechanism to select the most likely value from the MSP, enabling the resolution of complex co-references without hard-coded knowledge graphs.

Experimental Results

The model was evaluated on the MultiWOZ 2.1 and 2.2 benchmarks, the industry standards for complex, multi-domain dialogues.

  • State-of-the-Art Performance: The MSP-Large model achieved a Joint Goal Accuracy (JGA) of 57.2% on MultiWOZ 2.1 and 57.7% on MultiWOZ 2.2, outperforming established models like Trippy and AG-DST.
  • Error Correction and Indirect Mentions: The MSP module correctly tracked indirect mentions 91% of the time. In head-to-head tests, it successfully corrected previous mistakes 10% more frequently than standard "changed state" models.
  • Efficiency: The MSP-Base model (110M parameters) demonstrated high performance-per-parameter, matching the results of models twice its size (e.g., Seq2Seq-DU at 220M parameters).

Discussion and Conclusion

By mimicking human linguistic behavior—where interpretations are constantly re-analyzed against new evidence—the MSP provides a robust "workspace" for dialogue tracking. While the model still faces challenges with deep semantic logic (e.g., choosing between two different times mentioned in a single turn) and precise span extraction for long addresses, it represents a significant step toward more reliable and efficient task-oriented systems. Future work will focus on integrating structural grammar knowledge to further refine the model's understanding of complex sentence relationships.

Summary updated

Abstract

Dialogue state tracking (DST) is a component of the task oriented dialogue system. It is responsible for extracting and managing slots, where each slot represents a part of the information to accomplish a task, and slot value is updated recurrently in each dialogue turn. However, many DST models cannot update slot values appropriately. These models may repeatedly inherit wrong slot values extracted in previous turns, resulting in the fail of the entire DST task. They cannot update indirectly mentioned slots well, either. This study designed a model with a mentioned slot pool (MSP) to tackle the update problem. The MSP is a slot specific memory that records all mentioned slot values that may be inherited, and our model updates slot values according to the MSP and the dialogue context. Our model rejects inheriting the previous slot value when it predicates the value is wrong. Then, it extracts the slot value from the current dialogue context. As the contextual information accumulates, the new value is more likely to be correct. It also can track the indirectly mentioned slot by picking a value from the MSP. Experimental results showed our model reached state of the art DST performance on MultiWOZ datasets.
BibTeX
@inproceedings{sun2022tracking,
  author = {Zhoujian Sun and Zhengxing Huang and Nai Ding},
  title = {{On Tracking Dialogue State by Inheriting Slot Values in Mentioned Slot Pools}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2022},
  url = {https://openreview.net/forum?id=-TnimbQ4tFg},
}
Figure