Abstract
Task-oriented dialog systems increasingly rely on deep learning-based slot filling models, usually needing extensive labeled training data for target domains. Often, however, little to no target domain training data may be available, or the training and target domain schemas may be misaligned, as is common for web forms on similar websites. Prior zero-shot slot filling models use slot descriptions to learn concepts, but are not robust to misaligned schemas. We propose utilizing both the slot description and a small number of examples of slot values, which may be easily available, to learn semantic representations of slots which are transferable across domains and robust to misaligned schemas. Our approach outperforms state-of-the-art models on two multi-domain datasets, especially in the low-data setting.
Introduction
Scaling Task-Oriented Dialog (TOD) systems to new domains is traditionally a data-intensive process. Slot filling—the task of extracting parameters like destination_city or departure_date from user utterances—typically requires large volumes of domain-specific labeled data. This creates two significant challenges: the "Cold Start" problem, where no data exists for a new domain, and "Schema Misalignment," where identical slot names (e.g., depart) refer to different concepts (e.g., a city vs. a date) across different platforms.
While existing zero-shot models leverage natural language descriptions of slots to bridge these gaps, descriptions alone are often insufficient to resolve ambiguities in misaligned schemas. This paper introduces a robust model that combines slot descriptions with a few representative example values to create more discriminative semantic representations.
Methodology
The proposed architecture conditions the slot filling process on both high-level descriptions and concrete examples.
The model consists of three primary components:
- Encoders: A Bidirectional GRU (Bi-GRU) processes the user utterance, while a mean-pooling layer encodes the slot description.
- Slot Example Encoder: This component uses an attention mechanism to compare every word in the user's sentence against a set of provided example values (e.g., "London", "Paris"). This allows the model to identify semantic similarities between the input and known slot types.
- The Tagger: A Bidirectional LSTM (Bi-LSTM) integrates the word embeddings, description embeddings, and the "attended" example representations to predict IOB (Inside-Outside-Beginning) tags.
By incorporating example values, the model learns the type of data expected for a slot, providing a robust signal that complements potentially ambiguous slot names or descriptions.
Experimental Evaluation
The model was evaluated on two distinct datasets:
- SNIPS: A standard NLU benchmark with 39 slots across 7 intents. Cross-domain transfer was tested by training on 6 intents and evaluating on the held-out 7th intent.
- XSchema: A novel dataset designed to simulate real-world web schema misalignment. It incorporates schemas from various travel and booking sites where slot naming conventions vary significantly.
Results and Discussion
The experimental results demonstrate that the inclusion of example values significantly enhances zero-shot performance. On the SNIPS dataset, the model achieved a 2.9% absolute gain in F1 score over previous zero-shot baselines. The impact was even more pronounced on the XSchema dataset, with improvements of up to 10% F1 on complex intents like FindFlights.
A key insight from the study is the "less is more" phenomenon in zero-shot settings. As shown in the figure above, providing just two example values per slot often yields optimal performance in the zero-shot (0 instances) regime. Providing too many examples in this context can lead to overfitting on those specific values, which degrades the model's ability to generalize to new instances of the slot.
Conclusion
This work demonstrates that combining slot descriptions with a small set of example values creates a robust framework for zero-shot slot filling. This approach effectively mitigates the challenges of schema misalignment and data scarcity. Future research will focus on improving inference efficiency—currently, the model processes slots independently—and exploring joint modeling techniques to handle overlapping predictions and large slot sets more effectively.
Summary updated