Abstract
Spoken Language Understanding (SLU) typically involves two critical tasks: Intent Detection and Slot Filling. While joint models have improved performance by training these tasks together, they often rely solely on high-level semantic hidden states, potentially losing fine-grained lexical information. We present WAIS (Word Attention for Joint Intent Detection and Slot Filling), a framework that adaptively fuses raw word-level embeddings with semantic-level hidden states. By employing a novel fusion gate, WAIS achieves state-of-the-art results on the ATIS and Snips benchmarks, demonstrating the value of preserving raw lexical signals in joint SLU tasks.
Introduction
In the field of Spoken Language Understanding (SLU), systems are tasked with parsing user utterances into structured semantic frames. This involves identifying the user's goal (Intent Detection) and extracting relevant entities (Slot Filling). For example, in the phrase "Book a flight to Chicago," the intent is "BookFlight" and the slot is "Destination: Chicago."
Traditionally, these tasks were treated as independent components in a pipeline, which often led to error propagation. Recent advancements have shifted toward "joint models" that learn both tasks simultaneously using attention mechanisms on neural network hidden states. However, these models frequently overlook the raw, word-level information present in the original input. The WAIS framework addresses this by arguing that hidden states, while capturing context, may lose specific nuances that raw word embeddings can provide.
Methodology
The WAIS framework is built upon a Bidirectional Long Short-Term Memory (BLSTM) architecture, enhanced by two primary innovations:
Word-Level Attention
Standard attentional models typically compute context vectors based on the hidden states of the encoder. WAIS introduces an additional attention mechanism that operates directly on the source word embeddings. This produces a "word context vector" that preserves raw lexical information, ensuring that specific keywords are not diluted by the encoding process.
The Fusion Gate
To effectively combine semantic and lexical information, WAIS utilizes an adaptive fusion gate ($\lambda$). Rather than a simple summation, this gate learns to dynamically weight the contribution of the word-level context versus the semantic-level context for each input. This allows the model to prioritize raw keywords when they are most informative and rely on broader semantic flow when syntax is more critical. The resulting global context vector is then used for simultaneous intent and slot prediction.
Experimental Setup
The model was evaluated on two industry-standard datasets:
- ATIS (Airline Travel Information System): A domain-specific dataset focused on flight reservations, consisting of 4,978 training and 893 test utterances.
- Snips: A larger, multi-domain dataset containing 13,784 training and 700 test utterances, covering categories such as music, weather, and restaurant bookings.
Results
The WAIS model demonstrated superior performance compared to established baselines, including the attentional model by Liu and Lane (2016).
As shown in the table above, WAIS achieved consistent improvements across both metrics (Slot F1 and Intent Accuracy) on both datasets. On the ATIS dataset, Slot Filling F1 reached 95.87% and Intent Accuracy reached 98.6%. On the more complex Snips dataset, the improvements were even more pronounced, with Slot Filling F1 rising to 91.09% and Intent Accuracy to 98.43%. These results suggest that the fusion of word-level and semantic-level attention provides a more robust representation for SLU tasks, particularly in diverse, large-scale datasets.
Conclusion
WAIS bridges the gap between raw lexical data and high-level semantic representations in spoken language understanding. By introducing word-level attention and an adaptive fusion gate, the model successfully mitigates the information loss inherent in standard encoding processes. The state-of-the-art performance on ATIS and Snips confirms that a "forest and trees" approach—balancing global context with local word signals—is essential for optimal joint intent detection and slot filling.
Summary updated