Conversation disentanglement is the challenging task of partitioning a multi-party chat log into separate, coherent threads or "sessions." In environments like IRC, Slack, or Discord, multiple topics often overlap, making it difficult for automated systems to follow a single line of reasoning. This paper, presented at EMNLP 2022, introduces Bi-CL (Bi-Level Contrastive Learning), a framework designed to address these complexities by modeling conversations at both the individual utterance and global session levels.
The Challenge of Entangled Conversations
Existing disentanglement methods typically rely on a two-step "pairwise" approach: first predicting if one message is a reply to another, and then clustering these pairs into threads. However, this approach suffers from two primary limitations:
- Local Myopia: By focusing solely on pairs, models often miss the broader semantic context of a session.
- Data Scarcity: Pairwise models require extensive human-annotated "reply-to" labels, which are costly and time-consuming to produce.
Methodology: Bi-Level Contrastive Learning
The proposed Bi-CL framework utilizes a hierarchical encoding structure to capture both local and global dependencies.
The core of the model lies in its dual contrastive objectives:
- Utterance-Level Contrastive Loss ($L_u$): This objective pulls representations of messages belonging to the same session closer together in the embedding space while pushing messages from different sessions apart.
- Session-Level Contrastive Loss ($L_s$): This "bi-level" innovation maintains "prototypes" (centroids) for each session. It encourages individual messages to align with their respective session prototypes, effectively capturing the global semantic "vibe" of the thread.
For unsupervised scenarios, Bi-CL employs an Expectation-Maximization (EM) framework. It iteratively clusters messages using K-Means to identify "pseudo-prototypes" (E-step) and then updates the model to pull messages toward these centers (M-step). Additionally, a sub-network predicts the number of distinct threads ($K$) within a text block to guide the clustering process.
Evaluation and Results
The model was evaluated on two benchmark datasets: Ubuntu IRC, which contains complex technical support logs with high concurrency, and Movie Dialogue, a more controlled dataset of intermingled movie script lines.
As shown in the results table, Bi-CL achieved state-of-the-art (SOTA) performance across multiple metrics, including Normalized Mutual Information (NMI), Adjusted Rand Index (ARI), and the Shen-F score. Notably:
- In the supervised setting on Movie Dialogue, Bi-CL reached a Shen-F score of 0.747, significantly surpassing previous models like StructBERT.
- In the unsupervised category, Bi-CL dominated the Ubuntu IRC benchmark, achieving an ARI of 0.345, nearly doubling the performance of the "Co-Training" baseline.
Ablation studies confirmed that the session-level loss ($L_s$) is critical for performance, as its removal led to a significant drop in clustering quality.
Conclusion and Future Directions
Bi-CL demonstrates that modeling global session structures through contrastive learning is a powerful alternative to traditional pairwise methods. The small performance gap between supervised and unsupervised versions suggests that the inherent semantic structure of conversation threads is highly informative.
Future work may involve integrating this bi-level loss with larger Transformer-based architectures like BERT or GPT and addressing the challenges of handling extremely long-range dependencies in massive chat logs.
Summary updated