TL;DR
This paper introduces a novel Knowledge-Enhanced Multifactor Graph (KEMG) model for Dialogue Intent Classification (DIC). By integrating domain-specific knowledge graphs and modeling dialogues as a heterogeneous graph with multiple node types (utterances, speakers, local discourses), KEMG significantly improves intent recognition accuracy compared to existing GNN-based methods.
What Problem Does This Paper Address?
Dialogue Intent Classification (DIC) is crucial for understanding user needs in conversational AI. While Graph Neural Networks (GNNs) have shown promise, they often overlook valuable domain-specific knowledge, limiting their ability to capture fine-grained semantic nuances. Existing methods struggle to effectively model the complex relationships between utterances, speakers, and discourse structures within a dialogue, especially when these relationships are not contiguous. This paper aims to bridge this gap by incorporating external knowledge and a richer graph representation to enhance DIC.
Methods
The proposed KEMG model consists of two main components:
- Knowledge-Aware Utterance Encoder: This module enriches utterance representations by fusing token-level information with entity-level semantics extracted from a domain-specific knowledge graph (KG). It uses a KG embedding transformation to represent entities and then aligns and stacks token and entity embeddings. This combined representation is fed into a CNN+LSTM sequential encoder to produce knowledge-aware utterance representations.
- Heterogeneous Multifactor Dialogue GNN: This component constructs a multifactor graph for each dialogue. It treats utterances as "utterance nodes," introduces "speaker nodes" to capture speaker context, and adds "local discourse nodes" to aggregate consecutive utterances from the same speaker. These nodes are connected by five types of edges (speaker, local, local-speaker, utterance-order, local-order) to model temporal and hierarchical dependencies. A Relational Graph Convolutional Network (R-GCN) is then applied to propagate information across this heterogeneous graph. Finally, pooled features from all node types are concatenated to predict the dialogue intent.
Datasets
The authors evaluate their model on a dataset collected from JD.com's online customer service. This dataset comprises 20,000 dialogues, with 437,060 utterances, annotated with one of 50 intent labels. The dataset is split into 18,000 for training, 1,000 for validation, and 1,000 for testing.
Key Insights and Ideas
- Domain Knowledge is Key: Explicitly incorporating domain-specific knowledge graphs significantly boosts performance in dialogue intent classification, suggesting that understanding the entities and their relationships within a domain is critical.
- Multifaceted Dialogue Representation: Modeling dialogues not just as sequences of utterances but as a heterogeneous graph with distinct nodes for utterances, speakers, and local discourse segments captures richer contextual information.
- Hierarchical and Temporal Dependencies Matter: The various edge types in the multifactor graph effectively model the complex interplay between speakers, consecutive turns, and the overall dialogue flow, which is crucial for accurate intent prediction.
Key Results
The KEMG model achieves state-of-the-art performance on the JD customer service dataset. It outperforms other GNN-based baselines like DialogueGCN, RGAT, and DAG across several metrics, including Accuracy (Acc), Top-3 accuracy (Top-3), Top-5 accuracy (Top-5), Macro-F1 (M-F1), and Weighted-F1 (W-F1). For instance, KEMG achieved an accuracy of 67.70%, compared to the next best model (DAG) at 63.50%. The paper also shows that removing the knowledge-aware utterance encoder (KEMG-w/o KG) leads to a performance drop, confirming the positive impact of the domain-specific KG.
Limitations and Future Work
The authors acknowledge that their work focuses on single-label classification. They suggest that future work could extend KEMG to handle multi-label classification tasks, which would be a valuable advancement for more complex conversational scenarios.
Summary updated