Abstract
Dialogue state tracking is an essential part of goal-oriented dialogue systems, while most of these state tracking models often fail to handle unseen services. In this paper, we propose SGD-QA, a simple and extensible model for schema-guided dialogue state tracking based on a question answering approach. The proposed multi-pass model shares a single encoder between the domain information and dialogue utterance. The domain's description represents the query and the dialogue utterance serves as the context. The model improves performance on unseen services by at least 1.6x compared to single-pass baseline models on the SGD dataset. SGD-QA shows competitive performance compared to state-of-the-art multi-pass models while being significantly more efficient in terms of memory consumption and training performance.
Introduction
Building virtual assistants that scale across diverse services is a significant challenge in Dialogue State Tracking (DST). Traditional models are often "ontology-specific," relying on a fixed set of slots and values. This approach fails when encountering "unseen" services—new domains or APIs not present during training. To address this, researchers utilize Schema-Guided datasets where models are provided with natural language descriptions of services. While multi-pass models generalize well to these new schemas, they are often computationally expensive and slow. SGD-QA addresses these issues by framing DST as a streamlined Question Answering task.
Methodology: DST as Question Answering
The core innovation of SGD-QA is framing DST as a Question Answering (QA) task. Instead of predicting slots from a fixed list, the model treats the schema description (e.g., "the number of people for the reservation") as a query and the dialogue turn as the context.
Single Shared Encoder Architecture
Unlike previous multi-pass architectures (like SPPD) that use separate encoders for different components, SGD-QA employs a single shared BERT encoder. This design allows the model to learn a unified representation for intents, requested slots, and slot values, significantly reducing the parameter footprint.
The architecture includes specialized heads:
- Classification heads for binary (intent activation) and 3-way tasks (slot status: active, none, or "dontcare").
- Span prediction head for extracting specific text values from the dialogue, similar to SQuAD-style extraction.
To optimize training, the authors used data balancing to manage the high ratio of negative examples and automatic mixed precision for faster computation. This allows the model to converge in just 3 epochs, a massive improvement over the 80–160 epochs required by single-pass baselines.
Evaluation and Results
The model was evaluated on the Schema-Guided Dialogue (SGD) dataset, the largest public DST dataset with over 20,000 dialogues across 20 domains. The evaluation focuses on zero-shot generalization to unseen services.
Performance and Efficiency
SGD-QA demonstrates a significant performance leap, outperforming single-pass baselines by at least 1.6x on unseen services. In single-domain tasks, it achieved a Joint Goal Accuracy (Joint GA) of 65%, a 1.3x improvement over the standard SGD baseline.
Beyond accuracy, SGD-QA is remarkably efficient. By consolidating tasks into a single encoder, it reduces memory consumption to approximately 1/5th or 1/6th of previous multi-pass models like SPPD. Furthermore, because the queries are independent, they can be parallelized during inference to maintain low latency.
Discussion and Conclusion
The success of SGD-QA highlights the power of framing DST as a reading comprehension task. By focusing on the description of a slot rather than its identity, the model generalizes effectively to new domains. While challenges remain—such as the inherent difficulty of extractive tasks compared to classification—SGD-QA sets a new standard for efficient, scalable, and generalizable dialogue state tracking.
Summary updated