Simple, Fast, Accurate Intent Classification and Slot Labeling

Arshit Gupta, John Hewitt, Katrin Kirchhoff

Introduction

Task-oriented dialogue systems, such as virtual assistants, rely on Spoken Language Understanding (SLU) to process user requests. SLU typically involves two concurrent tasks: Intent Classification (IC), which identifies the user's overall goal, and Slot Labeling (SL), which extracts specific entities from the utterance. While joint modeling using complex recurrent neural networks like LSTMs has been the standard, these models are often slow to train and deploy due to their sequential nature. This paper introduces a modular framework that decouples word contextualization from label dependencies, enabling the use of fast, parallelizable architectures like Convolutional Neural Networks (CNNs) without sacrificing accuracy.

Methodology

The authors propose a framework that breaks the joint IC+SL task into four distinct, plug-and-play components:

  1. Word Contextualization: Transforming words into context-aware vectors.
  2. Sentence Representation: Summarizing the utterance into a single vector.
  3. Intent Prediction: Classifying the overall intent.
  4. Slot Label Prediction: Tagging individual words with entity labels.
Figure 1: The proposed modular framework for joint Intent Classification and Slot Labeling.

To bridge the performance gap between fast non-recurrent models and slower recurrent ones, the authors introduce a "label-recurrent" decoder. Instead of using a heavy recurrent network for the entire input, they apply a tiny (10-dimensional) LSTM only to the sequence of previously predicted slot labels. This allows the computationally intensive word contextualization to be performed in parallel (e.g., via CNNs), while still capturing the sequential dependencies inherent in slot labels.

Experimental Setup

The models were evaluated on two standard benchmarks:

  • ATIS: A classic dataset of airline reservation requests with long utterances but short slot spans.
  • Snips 2017: A diverse crowdsourced dataset with shorter utterances but longer, more complex slot spans.

The study compared several word contextualization paradigms, including Feed-forward, Dilated CNNs, Self-attention (Transformers), and BiLSTMs.

Key Results and Insights

The experiments revealed that local context is the most critical inductive bias for Slot Labeling. While pure feed-forward models performed poorly, the addition of even a single convolutional layer led to dramatic improvements.

Figure 3: Performance comparison across different architectures, highlighting the impact of label recurrence on accuracy and inference latency.

Performance and Speed

  • State-of-the-Art Accuracy: The proposed CNN-5KERNEL-3L model with label recurrence achieved a state-of-the-art Slot Labeling F1 score of 92.30 on the Snips dataset, representing a 30% error reduction over previous benchmarks.
  • Efficiency: This CNN-based model operates at over 2x the inference speed of a comparable 2-layer LSTM and trains in less than half the time, making it ideal for low-latency production environments.
  • Label Recurrence as a "Cheat Code": Adding label recurrence provided massive accuracy boosts to non-recurrent models like CNNs with a negligible latency penalty (~1.3ms per utterance).
  • Inductive Biases: Self-attention models struggled on these datasets, likely because they lack the necessary inductive biases for the relatively small data sizes typical in SLU tasks.

Conclusion

This research demonstrates that decoupling architectural components allows for the creation of SLU models that are both faster and more accurate than traditional fully recurrent networks. By using a modular approach and lightweight label recurrence, the authors achieved state-of-the-art results on the Snips dataset while significantly reducing inference latency. Future work involves integrating large-scale pre-trained embeddings like BERT into this modular framework to further enhance performance.

Summary updated

BibTeX
@inproceedings{gupta2019simple,
  author = {Arshit Gupta and John Hewitt and Katrin Kirchhoff},
  title = {{Simple, Fast, Accurate Intent Classification and Slot Labeling}},
  booktitle = {SIGdial},
  year = {2019},
  doi = {10.18653/v1/w19-5906},
}
Figure