Abstract
Out-of-Domain (OOD) intent detection is vital for practical dialogue systems, and it usually requires considering multi-turn dialogue contexts. However, most previous OOD intent detection approaches are limited to single dialogue turns. In this paper, we introduce a context-aware OOD intent detection (Caro) framework to model multi-turn contexts in OOD intent detection tasks. Specifically, we follow the information bottleneck principle to extract robust representations from multi-turn dialogue contexts. Two different views are constructed for each input sample and the superfluous information not related to intent detection is removed using a multi-view information bottleneck loss. Moreover, we also explore utilizing unlabeled data in Caro. A two-stage training process is introduced to mine OOD samples from these unlabeled data, and these OOD samples are used to train the resulting model with a bootstrapping approach. Comprehensive experiments demonstrate that Caro establishes state-of-the-art performances on multi-turn OOD detection tasks by improving the F1-OOD score of over 29% compared to the previous best method.
Introduction
Practical dialogue systems must operate under an "open-world" assumption, where user requests frequently fall outside the system's predefined categories. Detecting these Out-of-Domain (OOD) intents is essential for maintaining system reliability and providing helpful fallback responses. While single-utterance OOD detection is well-researched, real-world interactions are inherently multi-turn. This context introduces two major hurdles: the accumulation of conversational noise (superfluous information) over long histories and the extreme scarcity of labeled OOD data for multi-turn contexts.
The Caro Framework
To address these challenges, we propose Caro (Context-aware OOD intent detection). Caro leverages the Information Bottleneck (IB) principle to filter conversational noise and a self-training mechanism to utilize unlabeled data.
Multi-View Information Bottleneck
Caro processes the dialogue history and the current utterance through a BERT encoder. To distinguish between essential intent-related features and conversational noise, the framework constructs two distinct views of the input:
- Global Pooling View ($v_1$): Captures a broad representation of the entire context.
- Adaptive Reception Field View ($v_2$): Uses a gating mechanism to focus dynamically on the most relevant parts of the history.
By applying a multi-view IB loss ($\mathcal{L}_{IB}$), the model maximizes the mutual information between these views while minimizing the inclusion of superfluous information that does not contribute to intent classification.
Noise Reduction and the IB Principle
The effectiveness of the IB principle in Caro is demonstrated by its ability to minimize the difference $I(x;z) - I(z;y)$, which represents the amount of non-task-related information retained in the latent representation $z$. Compared to other methods like InfoMax or standard Multi-View Information (MVI), Caro significantly reduces this "superfluous" information, leading to more robust intent boundaries.
Two-Stage Self-Training
To overcome the lack of labeled OOD samples, Caro employs a two-stage bootstrapping approach using unlabeled data:
- Stage 1 (Synthesis): The model is initially trained on In-Domain (IND) data and synthetic OOD samples generated via a Mixup strategy in the feature space.
- Stage 2 (Mining): The preliminary model scans unlabeled data to identify high-confidence OOD samples. These "mined" samples are then incorporated into the final training set, allowing the model to learn more precise decision boundaries between IND and OOD space.
Experimental Results
We evaluated Caro on the STAR dataset, a task-oriented benchmark with 150 intents. Caro achieved a state-of-the-art F1-OOD score of 54.75% on the STAR-Full dataset, representing a 29% improvement over the previous best baseline (ASS+MSP).
Ablation studies confirmed that the IB loss is particularly critical for long-context dialogues (6+ turns), where conversational noise is most prevalent. Removing the IB component resulted in a nearly 10-point drop in performance for these complex scenarios, highlighting the importance of selective information retention.
Conclusion
Caro provides a principled framework for multi-turn OOD detection by combining information-theoretic noise reduction with effective self-training on unlabeled data. While currently focused on text, future work could extend these principles to multi-modal contexts, such as filtering background noise in voice-activated systems to better isolate user intent across different data streams.
Summary updated