Introduction
Dialogue State Tracking (DST) serves as the core "memory" component in task-oriented dialogue systems, responsible for maintaining a representation of the user's goals throughout a conversation. As dialogue systems evolve from single-domain tasks to complex multi-domain assistants, two primary challenges have emerged: slot interrelation and the Out-of-Vocabulary (OOV) problem. Existing models often struggle to capture dependencies between related slots across different domains (e.g., an "area" in a restaurant search influencing an "area" in a hotel search) and fail to handle entities not seen during training.
The GeeX Architecture
To address these limitations, we introduce GeeX (Generation and Extraction combined), a novel DST framework that integrates hierarchical ontology modeling with a hybrid decoding strategy.
The architecture consists of four key components:
- Masked Hierarchical Attention: Unlike traditional flat slot representations, GeeX models the dialogue ontology as a three-layer hierarchy (Domain → Intent → Slot). A masked attention mechanism facilitates information flow across these levels, allowing the model to learn shared semantics between related slots in different domains.
- Transformer Encoder: This component fuses the dialogue context—comprising the current user utterance and the previous system response—with the hierarchical state information.
- Operation Gate: For each slot, this gate predicts one of four operations:
CARRYOVER(retain previous value),NULL(slot not mentioned),GENERATE(use generative decoder), orEXTRACT(use extractive decoder). - Hybrid State Value Decoder: GeeX employs parallel decoders. The Generation Decoder handles abstract values using a GRU with a copy mechanism, while the Extraction Decoder identifies value spans directly within the dialogue text, functioning similarly to a reading comprehension model.
Experimental Evaluation
GeeX was evaluated on three prominent multi-domain benchmarks: MultiWOZ 2.0, MultiWOZ 2.1, and CrossWOZ. These datasets provide a rigorous testing ground for multi-domain complexity and cross-lingual applicability (English and Chinese).
Results and Discussion
GeeX achieves state-of-the-art performance across all evaluated datasets, demonstrating the effectiveness of its hybrid approach.
Key findings include:
- Superior Accuracy: On MultiWOZ 2.0, GeeX reached a Joint State Accuracy of 56.35%, outperforming the SOM baseline (52.32%). Similar gains were observed on CrossWOZ, where it achieved 54.70% accuracy.
- OOV Robustness: Stress tests involving masked tokens revealed that GeeX is significantly more resilient to OOV words than purely generative models. The extractive branch acts as a "safety net," allowing the model to capture specific entities directly from the text even when they are absent from the training vocabulary.
- Complementary Decoders: The study confirms that generation and extraction are complementary; extraction excels at long, specific sequences like addresses, while generation handles implied values.
Conclusion
GeeX demonstrates that combining hierarchical ontology modeling with a hybrid decoding strategy significantly improves DST performance and robustness. Future work will explore reducing dependency on large labeled datasets through few-shot learning and integrating external knowledge bases to further enhance flexibility in real-world applications.
Summary updated