Focus on Interaction: A Novel Dynamic Graph Model for Joint Multiple Intent Detection and Slot Filling

Zeyuan Ding, Zhihao Yang, Hongfei Lin, Jian WangIJCAI 2021 · DOI

TL;DR

This paper introduces a novel Dynamic Graph Model (DGM) for joint multiple intent detection and slot filling in spoken language understanding. By constructing a sentence-level intent-slot interactive graph that dynamically updates, DGM effectively models the relationship between intents and slots while mitigating noise and error propagation, achieving state-of-the-art results and significantly improving inference speed.

What Problem Does This Paper Address?

Spoken Language Understanding (SLU) systems are crucial for task-oriented dialogue. They typically involve two core tasks: intent detection (understanding the user's goal) and slot filling (extracting specific information or constraints). While joint models that tackle both tasks simultaneously generally outperform pipeline approaches, existing methods struggle with utterances expressing multiple intents. These models often introduce "intent noise" by treating all tokens as equally relevant to all intents, degrading performance. Furthermore, previous graph-based approaches for multi-intent scenarios construct token-level graphs, which can be computationally expensive and still suffer from noise. This paper aims to address these limitations by proposing a dynamic, sentence-level graph that selectively incorporates intent information only for relevant tokens, thereby improving accuracy and efficiency.

Methods

The proposed Dynamic Graph Model (DGM) consists of three main components:

  • Shared Encoder: A BiLSTM followed by a self-attention mechanism to capture contextual information from the input utterance.
  • Intent Decoder: Treats intent detection as a multi-label classification problem. It uses self-attention on the encoded utterance to predict multiple intents.
  • Dynamic Graph Module: This is the core innovation. It comprises:
  • Graph Generation Layer: This layer dynamically constructs a sentence-level intent-slot interactive graph. It calculates a "relevance score" between each intent and token. Tokens with a score above a threshold are considered important for that intent and are connected in the graph. This selective connection is key to reducing noise.
  • Graph Attention Network (GAT): A GAT is used to encode the generated intent-slot interactive graph. GATs are effective at learning relationships between nodes in a graph by assigning attention weights, allowing the model to capture nuanced interactions between intents and slots.
  • Slot Filling Decoder: A unidirectional LSTM that uses the output of the GAT and previous predictions to generate the slot labels.

The model is trained jointly to optimize both intent detection and slot filling objectives.

Datasets

The paper evaluates the DGM on both multi-intent and single-intent datasets:

  • Multi-Intent Datasets:
    • MixSNIPS
    • MixATIS (These datasets are described as having utterances with 1-3 intents.)
  • Single Intent Datasets:
    • ATIS
    • SNIPS

Key Insights and Ideas

  • Sentence-Level Graph for Efficiency: Instead of building a graph for each token (as in prior work), DGM builds a single graph for the entire sentence. This significantly reduces computational overhead and leads to a substantial speedup (3-6x).
  • Dynamic Relevance-Based Graph Construction: The model doesn't connect every token to every intent. It dynamically identifies and connects tokens that are truly relevant to specific intents, effectively filtering out noise and focusing on meaningful interactions.
  • Intent-Slot Interaction is Crucial: The paper emphasizes that modeling the interaction between intents and slots is more effective than treating them independently or using simpler joint methods. The graph structure explicitly captures this interplay.
  • GAT for Graph Encoding: Using Graph Attention Networks (GATs) proves more effective than Graph Convolutional Networks (GCNs) for encoding the intent-slot interactive graph, highlighting the importance of attention mechanisms in capturing complex relationships.

Key Results

  • State-of-the-Art Performance: DGM achieves state-of-the-art (SOTA) results on both multi-intent and single-intent datasets, demonstrating its effectiveness across different scenarios.
  • Significant Speedup: The model achieves a 3-6x speedup compared to the SOTA model (AGIF) on the MixATIS dataset, making it much more practical for real-world applications. For example, DGM took 6,413s while AGIF took 33,528s on MixATIS.
  • Improved Overall Accuracy: On multi-intent datasets like MixATIS and MixSNIPS, DGM shows improvements in overall accuracy (sentence-level semantic frame parsing) of 2.6% and 1.6% respectively.
  • Generalization: The model performs well on single-intent datasets, indicating its robustness and ability to generalize beyond multi-intent scenarios.

Limitations and Future Work

  • Limitations: The paper acknowledges that while DGM reduces noise compared to previous methods, the concept of "intent noise" can still be a challenge in complex multi-intent scenarios. The dynamic graph construction relies on a hyperparameter a to determine relevance, which might require tuning.
  • Future Work: The authors suggest that their dynamic graph model can be extended to other NLP tasks that involve modeling complex relationships between different linguistic units. The code for their model is publicly available, encouraging further research in this area.

Summary updated

BibTeX
@inproceedings{ding2021focus,
  author = {Zeyuan Ding and Zhihao Yang and Hongfei Lin and Jian Wang},
  title = {{Focus on Interaction: A Novel Dynamic Graph Model for Joint Multiple Intent Detection and Slot Filling}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2021},
  doi = {10.24963/ijcai.2021/523},
}
Figure