Modeling Non-Linear Structure in Multi-Party Dialogue

95 concepts90 papers1 proposal

When we read a book or a news article, text flows in a single, predictable line. Even in a standard two-person (dyadic) conversation, turns alternate back and forth like a tennis match. But step into a busy group chat, a corporate meeting, or an online forum, and this linear illusion shatters.

Multi-party conversations (MPCs) are fundamentally non-linear. In a group setting, multiple participants interact simultaneously, creating a "who-said-what-to-whom" challenge characterized by overlapping threads and complex social dynamics. As evaluated by Chao-Hong Tan et al. in Is ChatGPT a Good Multi-Party Conversation Solver?, standard language models frequently struggle to maintain a coherent understanding of these interactions because their internal attention mechanisms are optimized for flat, sequential text.

The Geometry of Entanglement

To understand why standard natural language processing models fail in group settings, consider the geometry of a conversation. In a dyadic chat, turn $T_n$ is almost always a direct response to turn $T_{n-1}$. In an MPC, however, turn $T_n$ might reply to an utterance $T_{n-5}$ made by a speaker across the room, while turns $T_{n-4}$ through $T_{n-1}$ belong to a completely separate parallel sub-thread.

In Sentence Dependency Tagging in Online Question Answering Forums, Yang Liu et al. describe these relationships as a web-like structure of non-sequential dependencies. When a sequence-to-sequence model attempts to process this web as a linear progression, it suffers from severe context fragmentation. As Yuejie Lei et al. demonstrate in A Finer-grain Universal Dialogue Semantic Structures based Model For Abstractive Dialogue Summarization, treating dialogue linearly causes critical details to become scattered across non-adjacent turns, leading to a significant drop in performance for downstream tasks like abstractive summarization.

This structural blindness is not easily fixed by simply expanding the model's context window. In Exploring the Potential of ChatGPT on Sentence Level Relations: A Focus on Temporal, Causal, and Discourse Relations, Chunkit Chan et al. found that even advanced large language models struggle to accurately parse the discourse links of multi-party conversations, occasionally performing close to random guessing when tasked with mapping distant relational connections.

Mapping Conversations as Graphs

To capture the true topology of group interactions, researchers have shifted from linear sequence modeling to graph-based architectures. By representing a conversation as a graph, models can explicitly encode the relationships between disparate utterances and speakers, bypassing the artificial constraint of chronological order.

Linear Sequence (Fails to capture threads):
[Turn 1: Alice] ──> [Turn 2: Bob] ──> [Turn 3: Charlie] ──> [Turn 4: Alice]

Graph Representation (Captures true dependencies):
[Alice] 👤 <──────────────────────────────┐ (Speaker Relation)
   │                                      │
[Turn 1] 💬 ─── (Parallel Thread) ───> [Turn 4] 💬
   │
   └─── (Direct Reply) ───> [Turn 2] 💬 👤 [Bob]

A prominent example of this approach is the HeterMPC framework introduced by Jia-Chen Gu et al. in HeterMPC: A Heterogeneous Graph Neural Network for Response Generation in Multi-Party Conversations. Instead of treating the chat log as a uniform stream of text, HeterMPC constructs a heterogeneous graph containing two distinct node types: utterances and speakers. Edges are drawn based on interaction dynamics—such as who spoke after whom, and who addressed whom. This allows the network to model multi-directional proposals and track whether a message is intended for an individual, a specific sub-group, or the entire collective.

Similarly, the authors of DADgraph: A Discourse-aware Dialogue Graph Neural Network for Multiparty Dialogue Machine Reading Comprehension (Jiaqi Li et al.) utilize a discourse-aware graph to explicitly map functional relationships across entangled chat logs. By embedding these structural paths directly into the neural architecture, the model can trace rhetorical links back through time, successfully identifying the context of an utterance even when it references statements made dozens of turns prior.

Disentanglement and Thread Detection

Before a model can reason over a graph, it must first solve a fundamental preprocessing problem: conversation disentanglement. In a high-density chat environment, multiple independent conversations are often interleaved chronologically.

The scale of this challenge was quantified by Jonathan K. Kummerfeld et al. in A Large-Scale Corpus for Conversation Disentanglement. Their analysis revealed that a vast majority of standard dialogue datasets are heavily contaminated with missing messages or overlapping noise, proving that models cannot safely assume a single, unified conversation thread.

To untangle these streams, Dakuo Wang et al. developed the Context-Aware Thread Detection (CATD) framework in Context-Aware Conversation Thread Detection in Multi-Party Chat. Instead of relying on shallow, hand-engineered features or simple message-to-message comparisons, CATD uses deep linguistic representations to evaluate the semantic "flow" of an entire evolving thread. By calculating how well a new message fits into the broader context of existing parallel tracks, the model can effectively separate a chaotic chat log into clean, independent conversation streams.

Adapting Autoregressive Models to Non-Linear Dialogue

While graph neural networks offer an elegant mathematical representation of non-linear dialogue, modern conversational AI relies heavily on autoregressive Transformers (such as LLMs) that are natively designed to predict tokens sequentially. Bridging the gap between non-linear discourse and linear token generation is an active area of innovation, primarily split into two strategies: structural prompting and latent variable modeling.

Structural Prompting

The most direct way to adapt sequential models is to explicitly inject structural metadata into the text stream. In Is ChatGPT a Good Multi-Party Conversation Solver?, Chao-Hong Tan et al. demonstrated that standard zero-shot prompting fails in complex group dynamics. However, by explicitly weaving speaker identities and intended addressee metadata directly into the context window—a technique called MPC Structure Incorporation—they significantly restored the model's ability to track information flow and maintain situational awareness.

Latent Variable Modeling

In many real-world scenarios, explicit metadata regarding "who is speaking to whom" is missing or unannotated. To overcome this, Yiyang Li et al. proposed an Expectation-Maximization (EM) framework in EM Pre-training for Multi-party Dialogue Response Generation. Instead of requiring manual annotations, their approach treats the target addressee as a hidden, latent variable. During the "E-step," the model estimates the probability distribution of the intended recipient based on the conversation history; during the "M-step," it optimizes response generation conditioned on that distribution. This iterative learning process allows the model to internalize the underlying branching reply trees of multi-party dialogue without requiring labor-intensive structural labels.

History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_multi_party_dialogue_complexity_6d66a59f,
  title = {multi-party dialogue complexity},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/multi-party-dialogue-complexity/rdr_6d66a59f85224f97b6746e100e35d243},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_multi_party_dialogue_complexity_6d66a59f_v1,
  title = {multi-party dialogue complexity},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/multi-party-dialogue-complexity/rdr_6d66a59f85224f97b6746e100e35d243/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake