Abstract
Conversation disentanglement aims to group utterances into detached sessions, which is a fundamental task in processing multi-party conversations. Existing methods have two main drawbacks. First, they overemphasize pairwise utterance relations but pay inadequate attention to the utterance-to-context relation modeling. Second, huge amount of human annotated data is required for training, which is expensive to obtain in practice. To address these issues, we propose a general disentangle model based on bi-level contrastive learning. It brings closer utterances in the same session while encourages each utterance to be near its clustered session prototypes in the representation space. Unlike existing approaches, our disentangle model works in both supervised setting with labeled data and unsupervised setting when no such data is available. The proposed method achieves new state-of-the-art performance on both settings across several public datasets.
The Challenge of Entangled Conversations
In platforms like Slack, Discord, or IRC, multiple conversations often happen simultaneously in a single channel. For a machine to understand these "entangled" chats, it must first group utterances into detached sessions. This task is known as conversation disentanglement.
Existing approaches usually suffer from two flaws:
- Local Myopia: They focus heavily on pairwise relations (e.g., "Does message B reply to message A?") but ignore the broader context of the entire session.
- Data Hunger: Most high-performing models require massive amounts of human-annotated data, which is expensive and time-consuming to produce.
Bi-Level Contrastive Learning (Bi-CL)
The core of the proposed framework is Bi-Level Contrastive Learning (Bi-CL). It uses a hierarchical architecture to encode utterances: first at the token level to capture individual message semantics, and then at the context level to capture how messages relate to their neighbors.
The "Bi-Level" aspect refers to two specific loss functions:
- Utterance-level Contrastive Loss: This pulls utterances from the same session closer together in the representation space while pushing utterances from different sessions apart.
- Session-level Contrastive Loss: The model maintains "prototypes" (centroids) for each session. It encourages every utterance to be near its assigned session prototype, capturing the global topic or "vibe" of the thread.
For the unsupervised setting, the authors employ an Expectation-Maximization (EM) framework. In the E-step, they use K-Means to estimate session assignments (clusters). In the M-step, they update the neural network to maximize the likelihood of these assignments.
Evaluation and Datasets
The model was evaluated on two major benchmarks:
- Ubuntu IRC: A dataset of technical support chats. It is highly complex, with up to 14 concurrent sessions and significant jargon.
- Movie Dialogue: A dataset created by intermingling sessions from movie scripts. It is generally cleaner but provides a good test for generalizability.
Key Insights
- Prototypes Matter: A message shouldn't just look like its neighbor; it should look like the entire thread. Introducing session prototypes allows the model to maintain a "global" memory of the conversation topic.
- Efficiency over Scale: The authors used a relatively lightweight Bi-LSTM rather than a massive Transformer like BERT. Despite this, their contrastive learning strategy allowed them to outperform several BERT-based baselines.
- The Power of EM: By framing unsupervised disentanglement as an EM problem, the model can "self-correct." As the embeddings get better, the clustering gets more accurate, which in turn provides a cleaner signal to further improve the embeddings.
Results
Bi-CL achieved the highest Shen-F scores (a metric for cluster quality) on both datasets. On Ubuntu IRC, it reached a Shen-F of 0.707, outperforming the previous best (StructBERT) at 0.681.
In the unsupervised setting, Bi-CL significantly outperformed existing co-training and zero-shot methods. On the Movie Dialogue set, its unsupervised performance (0.689 Shen-F) was remarkably close to its supervised performance (0.747), suggesting the model is highly robust without labels. Ablation studies confirmed that removing the session-level loss led to a noticeable drop in performance, highlighting the importance of global session structure.
Limitations and Future Work
While powerful, the authors acknowledge several areas for improvement:
- Backbone Constraints: Moving to a Transformer-based architecture (like BERT or RoBERTa) could potentially yield even higher accuracy.
- Predicting Session Count: Determining exactly how many sessions ($K$) are in a conversation remains a challenge. The model currently relies on external heuristics to estimate $K$.
- Long-range Dependencies: Real-world conversations can span hundreds of messages; testing the model's ability to handle very long-range thread dependencies is a key next step.
Summary updated