Generation and Extraction Combined Dialogue State Tracking with Hierarchical Ontology Integration

Xinmeng Li, Qian Li, Wansen Wu, Quanjun YinEMNLP 2021 · DOI

Introduction

Dialogue State Tracking (DST) serves as the core "memory" component in task-oriented dialogue systems, responsible for maintaining a representation of the user's goals throughout a conversation. As dialogue systems evolve from single-domain tasks to complex multi-domain assistants, two primary challenges have emerged: slot interrelation and the Out-of-Vocabulary (OOV) problem. Existing models often struggle to capture dependencies between related slots across different domains (e.g., an "area" in a restaurant search influencing an "area" in a hotel search) and fail to handle entities not seen during training.

The GeeX Architecture

To address these limitations, we introduce GeeX (Generation and Extraction combined), a novel DST framework that integrates hierarchical ontology modeling with a hybrid decoding strategy.

The GeeX architecture featuring hierarchical attention, a transformer encoder, an operation gate, and dual decoders.

The architecture consists of four key components:

  1. Masked Hierarchical Attention: Unlike traditional flat slot representations, GeeX models the dialogue ontology as a three-layer hierarchy (Domain → Intent → Slot). A masked attention mechanism facilitates information flow across these levels, allowing the model to learn shared semantics between related slots in different domains.
  2. Transformer Encoder: This component fuses the dialogue context—comprising the current user utterance and the previous system response—with the hierarchical state information.
  3. Operation Gate: For each slot, this gate predicts one of four operations: CARRYOVER (retain previous value), NULL (slot not mentioned), GENERATE (use generative decoder), or EXTRACT (use extractive decoder).
  4. Hybrid State Value Decoder: GeeX employs parallel decoders. The Generation Decoder handles abstract values using a GRU with a copy mechanism, while the Extraction Decoder identifies value spans directly within the dialogue text, functioning similarly to a reading comprehension model.

Experimental Evaluation

GeeX was evaluated on three prominent multi-domain benchmarks: MultiWOZ 2.0, MultiWOZ 2.1, and CrossWOZ. These datasets provide a rigorous testing ground for multi-domain complexity and cross-lingual applicability (English and Chinese).

Results and Discussion

GeeX achieves state-of-the-art performance across all evaluated datasets, demonstrating the effectiveness of its hybrid approach.

Performance comparison of GeeX against baseline models on MultiWOZ and CrossWOZ datasets.

Key findings include:

  • Superior Accuracy: On MultiWOZ 2.0, GeeX reached a Joint State Accuracy of 56.35%, outperforming the SOM baseline (52.32%). Similar gains were observed on CrossWOZ, where it achieved 54.70% accuracy.
  • OOV Robustness: Stress tests involving masked tokens revealed that GeeX is significantly more resilient to OOV words than purely generative models. The extractive branch acts as a "safety net," allowing the model to capture specific entities directly from the text even when they are absent from the training vocabulary.
  • Complementary Decoders: The study confirms that generation and extraction are complementary; extraction excels at long, specific sequences like addresses, while generation handles implied values.

Conclusion

GeeX demonstrates that combining hierarchical ontology modeling with a hybrid decoding strategy significantly improves DST performance and robustness. Future work will explore reducing dependency on large labeled datasets through few-shot learning and integrating external knowledge bases to further enhance flexibility in real-world applications.

Summary updated

BibTeX
@inproceedings{li2021generation,
  author = {Xinmeng Li and Qian Li and Wansen Wu and Quanjun Yin},
  title = {{Generation and Extraction Combined Dialogue State Tracking with Hierarchical Ontology Integration}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2021},
  doi = {10.18653/v1/2021.emnlp-main.171},
}
Figure