Abstract
Dialogue state tracking (DST) is a critical component of task-oriented dialogue systems, responsible for extracting and managing slot values from user utterances. Traditional models often suffer from error propagation due to "naïve inheritance" of previous states and struggle with indirect mentions. This study introduces a model utilizing a Mentioned Slot Pool (MSP)—a slot-specific memory that records all mentioned values. By allowing the model to reject previous values and re-extract them from updated context, the MSP approach significantly reduces inheritance errors. Experimental results demonstrate that the MSP model achieves state-of-the-art performance on the MultiWOZ 2.1 and 2.2 datasets while maintaining high parameter efficiency.
Introduction
In task-oriented dialogue systems, such as those used for booking travel or restaurants, Dialogue State Tracking (DST) serves as the engine that manages "slots" (e.g., destination, time, number of people). A persistent challenge in DST is the management of slot values across multiple turns. Most existing models rely on a simple inheritance mechanism: if a slot is not explicitly mentioned in the current turn, the model carries over the value from the previous turn.
This approach leads to two primary issues:
- Error Propagation: An early misinterpretation (e.g., misidentifying a date) is inherited turn after turn, leading to total task failure.
- Indirect Mentions: Users frequently refer to previous information implicitly (e.g., "the same day as my train"). Standard models often fail to map these cross-slot dependencies correctly.
Methods: The Mentioned Slot Pool (MSP)
The proposed architecture replaces static inheritance with a dynamic Mentioned Slot Pool (MSP). The MSP acts as a slot-specific memory, storing all values mentioned throughout the conversation.
The model processes each turn through a three-step pipeline:
- Hit Type Prediction: The model classifies each slot into one of four categories:
none,dontcare,mentioned(value exists in the MSP), orhit(value must be extracted from the current text). If ahitis predicted for an already-filled slot, the model effectively "rejects" the previous state and re-analyzes the context. - Mentioned Value Selection: For slots categorized as
mentioned, a bilinear model selects the most probable value from the MSP, facilitating the resolution of indirect references. - Hit Value Prediction: New values are extracted using either a classifier (for categorical slots) or a span-predictor (for open-ended text) based on the current dialogue context.
The system utilizes BERT (base and large) as the underlying language model to generate contextual embeddings.
Results and Discussion
The MSP model was evaluated on the MultiWOZ 2.1, MultiWOZ 2.2, and WOZ 2.0 benchmarks. As shown in the performance comparison, the MSP model achieved new state-of-the-art (SOTA) results among models not utilizing external data augmentation.
Key findings include:
- Performance: The MSP-L model reached 57.2% Joint Goal Accuracy (JGA) on MultiWOZ 2.1 and 57.7% on MultiWOZ 2.2.
- Efficiency: The MSP-B model (110M parameters) outperformed significantly larger models like AG-DST-A (680M parameters), demonstrating that intelligent memory management is more effective than sheer model scale.
- Error Reduction: Ablation studies confirmed that the MSP module is responsible for a 2-3% performance gain, primarily by reducing "Inherit Errors"—instances where the model incorrectly carries over a previous mistake.
Conclusion
By treating dialogue state as a dynamic memory rather than a static inheritance, the MSP model effectively breaks the cycle of error propagation in DST. While the model shows significant improvement, it still faces challenges with complex semantic reasoning, such as handling mid-sentence user corrections. Future research will focus on integrating external linguistic knowledge to address these deep semantic hurdles and further refine the "re-analysis" strategy.
Summary updated