Abstract
Dialog State Tracking (DST) is a critical component of task-oriented dialog systems, traditionally relying on large language models to memorize domain-specific schemas within their parameters. This paper introduces a retrieval-augmented framework that "grounds" the model by providing relevant domain knowledge from an external database. By decoupling language understanding from schema memorization, this approach significantly enhances performance, particularly in few-shot scenarios where training data is limited.
Introduction
In Task-Oriented Dialog (TOD) systems, Dialog State Tracking (DST) identifies user goals (e.g., "find a cheap hotel with parking"). Conventional methods map conversation history directly to states, leading to two primary issues: parameter entanglement, where the model must simultaneously learn natural language and domain-specific vocabulary, and data hunger, requiring extensive retraining for new domains. The authors propose that models should "look up" schema information rather than memorizing it, treating the external knowledge base as a "cheat sheet."
Methods: The Knowledge-Grounded Framework
The proposed framework utilizes a four-step pipeline to integrate external knowledge into the DST process:
- Encoding: An encoder generates representations for the dialog context and the Knowledge Base (KB), which contains slot types and slot-value pairs.
- Retrieval: Maximum Inner Product Search (MIPS) identifies the top-k most relevant knowledge elements.
- Integration: Retrieved elements are flattened into text and prepended to the user query, providing immediate context.
- Decoding: A standard decoder processes the augmented input to generate the final dialog state.
The retriever is optimized using a contrastive loss function to align query and ground-truth knowledge representations in vector space.
Experimental Setup
The method was evaluated on MultiWOZ 2.4, a standard benchmark for multi-domain TOD. Experiments covered two settings:
- Full-data: Using 100% of the training set.
- Few-shot: Using only 1% of the training data to simulate adapting to a new domain.
Results and Key Insights
The researchers found that providing specific slot-value pairs (e.g., hotel-parking: yes) was more effective than just providing slot types, despite the larger search space. The model also demonstrated robustness to "noisy" retrieval, successfully identifying correct information even when distractors were present in the top-k results.
As shown in the results table, the grounded approach offers substantial benefits:
- Few-Shot Advantage: With a T5-XXL backbone and 1% data, the "type+value" retrieval achieved a 55.32% Joint Goal Accuracy (JGA), outperforming the D3ST baseline (54.37%).
- Model Efficiency: The impact was most pronounced on smaller models. For T5-base at 1% data, the grounded approach reached 34.89% JGA, more than doubling the performance of D3ST (16.37%).
- Scalability: In the full-data setting, the model remained competitive with state-of-the-art methods (~75.5% JGA), indicating that grounding maintains performance even when data is abundant.
Discussion and Conclusion
While effective, the system faces challenges regarding retrieval sensitivity—where highly similar but incorrect elements cause errors—and the "closed-set" assumption, which complicates handling unique user-provided values not in the database. Future work will explore schema induction to allow models to build their own knowledge bases for entirely unseen domains. By shifting the burden of memorization to a retrieval system, knowledge-grounded DST provides a more flexible and data-efficient path for building robust conversational agents.
Summary updated