LCAN: A Label-Aware Contrastive Attention Network for Multi-Intent Recognition and Slot Filling in Task-Oriented Dialogue Systems

Shuli Zhang, Zhiqiang You, Xiao Xiang Qi, Peng Liu, Gaode Wu, Kan Xia, Shenguang HuangEMNLP 2025

TL;DR

This paper introduces LCAN, a novel network for understanding user requests in dialogue systems that involve multiple intents and slots within a single utterance. By combining label-aware attention and contrastive learning, LCAN significantly improves the accuracy of identifying both intents and their associated slots, especially in complex scenarios.

What Problem Does This Paper Address?

Task-oriented dialogue (TOD) systems, like virtual assistants, are becoming increasingly sophisticated. A key challenge they face is understanding user utterances that express multiple intentions and contain multiple pieces of information (slots) simultaneously. For example, a user might say, "Book a flight to London and find a hotel there." This single sentence contains two intents: "book flight" and "find hotel," along with slots like "destination: London." Traditional methods often struggle to disentangle these overlapping intents and slots, leading to errors in understanding and poor user experiences. This paper specifically tackles the problem of accurately recognizing multiple intents and filling their corresponding slots in such complex, multi-intent utterances.

Methods

The proposed LCAN (Label-Aware Contrastive Attention Network) is a joint modeling approach that integrates two core novelties:

  • Label-Aware Attention: This mechanism focuses on aligning specific input tokens with relevant intent and slot labels. It uses specialized attention layers to extract label-specific contextual representations, helping to disambiguate inputs and understand which parts of the utterance relate to which intent or slot. This is achieved through separate attention mechanisms for intents and slots, with a multi-layer approach for slots to handle their hierarchical nature.
  • Contrastive Learning: To address semantic overlap and ambiguity between similar intents and slots, LCAN employs contrastive learning. This technique works by creating "positive pairs" (similar examples) and "negative pairs" (dissimilar examples) to train the model. LCAN applies contrastive learning at three levels:
  • Intent Label Contrastive Learning: Encourages similar intent representations to be closer and dissimilar ones to be further apart.
  • Slot Label Contrastive Learning: Similar to intent learning, but for slot labels, helping to distinguish between semantically close slots.
  • Cross-Task Contrastive Learning: Enforces similarity between aligned intent-slot pairs, strengthening the connection between the two tasks.

These components are integrated into a unified model with a shared text encoder (RoBERTa-base), task-specific BiLSTM encoders, and finally, intent and slot decoders.

Datasets

The paper evaluates LCAN on two standard benchmark datasets for multi-intent NLU:

  • MixATIS: Primarily focused on the airline travel domain, containing intents like flight booking and inquiry.
  • MixSNIPS: A more diverse, open-domain dataset covering intents related to weather, music, restaurants, and movies, with a conversational style.

Key Insights and Ideas

  • The Power of Label-Specific Context: The label-aware attention mechanism is crucial for disentangling complex utterances. By explicitly linking parts of the input to specific labels, it helps the model understand the nuanced relationships that simple attention might miss.
  • Contrastive Learning for Ambiguity Resolution: Semantic overlap is a major hurdle in multi-intent scenarios. Contrastive learning provides a principled way to push apart representations of ambiguous or overlapping intents/slots, leading to more robust predictions.
  • Hierarchical Slot Handling: Slots often have a hierarchical structure. The multi-layer slot attention in LCAN acknowledges this, allowing for finer-grained understanding of slot information.
  • Joint Modeling Synergy: The paper strongly advocates for joint modeling, showing that by learning intent recognition and slot filling together, and by using cross-task contrastive learning, the model can achieve better overall performance than treating them separately.

Key Results

LCAN demonstrates significant improvements over state-of-the-art baselines on both datasets:

  • MixATIS: LCAN achieved a joint accuracy of 59.5%, outperforming MISCA by 1.4% and BiSLU by 8.0%. It also achieved a slot F1-score of 92.4%, 3.0% higher than BiSLU.
  • MixSNIPS: LCAN achieved a joint accuracy of 85.3%, outperforming BiSLU by 1.5%.
  • Ablation Studies: Removing contrastive learning modules led to a decrease of 1.2% to 2.2% in joint accuracy, highlighting the importance of contrastive learning for semantic consistency. Removing label-aware attention components also resulted in significant drops, particularly in slot filling accuracy.
  • Case Study: A qualitative analysis showed LCAN effectively disambiguating slots around conjunctions like "and," a common failure point for other models.

Limitations and Future Work

The authors acknowledge several limitations:

  • Parameter Count and Complexity: The integration of label-aware attention and contrastive learning increases model complexity and parameter count.
  • Sample Selection Bias: The effectiveness of contrastive learning depends on the design of positive/negative sample pairs, and semantic overlaps can introduce bias.
  • Word-Level vs. Label-Level Focus: The label-level focus of contrastive learning might overlook finer-grained semantic distinctions at the word or phrase level.

Future work could explore:

  • Hierarchical Attention Mechanisms: Further refining attention to capture more complex hierarchical structures.
  • Optimized Contrastive Strategies: Developing more sophisticated methods for generating positive/negative samples to improve generalization to diverse dialogues.
  • Fine-grained Semantic Similarity: Incorporating finer-grained semantic similarity computations to potentially yield further performance gains.

Summary updated

BibTeX
@inproceedings{zhang2025lcan,
  author = {Shuli Zhang and Zhiqiang You and Xiao Xiang Qi and Peng Liu and Gaode Wu and Kan Xia and Shenguang Huang},
  title = {{LCAN: A Label-Aware Contrastive Attention Network for Multi-Intent Recognition and Slot Filling in Task-Oriented Dialogue Systems}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2025},
}
Figure