TL;DR
This paper introduces Bi-CL, a bi-level contrastive learning framework that disentangles multi-party conversations into distinct threads by modeling both local utterance relationships and global session structures. By leveraging session "prototypes," the model achieves state-of-the-art performance in both supervised and unsupervised settings, significantly reducing the reliance on expensive human-annotated data.
What Problem Does This Paper Address?
In multi-party environments like IRC channels, Slack, or group chats, multiple conversations often happen simultaneously. Utterances from different topics become interleaved, making it difficult for both humans and machines to follow a specific thread—a challenge known as conversation disentanglement.
Existing solutions typically suffer from two major flaws:
- Local Myopia: Most models focus on pairwise relations (e.g., "Does message B reply to message A?"). This ignores the broader context of the entire session, leading to errors when the connection between two messages is subtle or depends on the overall topic.
- Data Hunger: High-performing models usually require massive amounts of human-labeled data (session IDs or "reply-to" links), which are incredibly tedious and expensive to produce.
Methods
The authors propose a Bi-level Contrastive Learning (Bi-CL) framework to address these gaps. The core idea is to learn an embedding space where utterances from the same session naturally "flock" together.
The Architecture
The model uses a hierarchical Bi-LSTM encoder. First, it encodes individual utterances; then, a context-level encoder allows these utterances to interact with their neighbors to capture temporal and conversational flow.
Bi-Level Losses
The "Bi-level" aspect refers to two distinct contrastive objectives:
- Utterance-level Contrastive Loss: This pulls utterances from the same session closer together while pushing utterances from different sessions apart.
- Session-level Contrastive Loss: The model introduces session prototypes (centroids in the embedding space). Utterances are encouraged to be near their assigned session prototype, capturing the global semantic structure of the dialogue.
Unsupervised Extension
For cases with no labeled data, the authors use an Expectation-Maximization (EM) approach. The model iteratively clusters utterances to estimate session assignments (E-step) and then updates the encoder parameters to minimize the contrastive losses based on those estimates (M-step).
Datasets
The model was evaluated on two major benchmarks:
- Ubuntu IRC: A dataset of technical support chats. It is highly complex, with a maximum of 14 concurrent sessions in the evaluated segments.
- Movie Dialogue: Created by manually intermingling sessions from movie scripts. It features more distinct narrative threads but fewer concurrent sessions (max 6).
Key Insights and Ideas
- Prototypes as Anchors: By forcing utterances to cluster around a central prototype, the model moves beyond simple pairwise matching. This helps maintain thread consistency even when a speaker changes the subject slightly or uses different vocabulary.
- The Power of EM: The unsupervised version of Bi-CL performs surprisingly close to the supervised version. This suggests that the inherent structure of conversations (who speaks when, and the semantic similarity of messages) is often enough for a model to "discover" threads on its own.
- Speaker Heuristics: In the unsupervised setting, the authors use a clever heuristic: they assume a single speaker rarely participates in multiple threads simultaneously within a short window. This provides enough "pseudo-labels" to kickstart the contrastive learning process.
Key Results
Bi-CL achieved new state-of-the-art (SOTA) results across almost all metrics.
- Supervised Performance: On the Movie Dialogue dataset, Bi-CL reached an NMI (Normalized Mutual Information) of 0.575, a massive jump from the previous SOTA of 0.448 (StructBERT).
- Unsupervised Performance: Even without labels, Bi-CL outperformed many supervised baselines. On Ubuntu IRC, its unsupervised Shen-F score (0.704) actually beat several supervised models like DialBERT and Transition.
- Robustness: Unlike many competitors that rely on dataset-specific features (like timestamps in IRC), Bi-CL relies primarily on text and speaker identity, making it more generalizable across different types of dialogue.
Limitations and Future Work
While Bi-CL is a significant step forward, the authors note several areas for improvement:
- Encoder Choice: The current model uses Bi-LSTMs for efficiency. Replacing these with large-scale pre-trained models like BERT could likely boost performance further.
- Session Count ($K$): The model currently needs to predict the number of sessions ($K$) in a conversation. While it uses a light-weight network for this, predicting $K$ accurately remains a bottleneck.
- Long-range Dependencies: The experiments were limited to segments of 50 utterances. Future work is needed to see how well these prototypes hold up in very long, sprawling conversations with dozens of active threads.
Summary updated