Who Says What to Whom: A Survey of Multi-Party Conversations

Jia-Chen Gu, Chongyang Tao, Zhen-Hua Ling

Abstract

Multi-party conversations (MPCs) involve complex, "many-to-many" interactions that diverge significantly from the sequential nature of traditional two-party dialogues. This survey provides a comprehensive overview of the MPC landscape, establishing a taxonomy that distinguishes between conversational context modeling—how systems perceive the graphical flow of information—and conversational component modeling—how they identify speakers, addressees, and content. By analyzing state-of-the-art methods and datasets, the paper highlights the shift toward graph-based representations and identifies key gaps for future research, such as unified modeling and high-level applications.

Introduction: The Complexity of Multi-Party Interaction

Most existing dialogue research focuses on Two-Party Conversations (TPCs), which follow a predictable, sequential "A-B-A-B" pattern. However, real-world communication—like Slack channels, Reddit threads, or business meetings—is rarely that simple. In Multi-Party Conversations (MPCs), the sequential flow breaks down. Multiple sub-conversations can happen simultaneously (interleaving), and a single message might reply to an utterance from many turns ago rather than the one immediately preceding it.

Graphical representation of a multi-party conversation showing non-linear information flow between interlocutors (I) and utterances (U).

The core challenge in MPC research is the "Who says What to Whom" problem: building AI that can navigate these non-linear, graphical information flows where speaker identification, addressee recognition, and thread disentanglement are non-trivial tasks.

A Taxonomy of MPC Understanding

The field is organized into two primary pillars: Conversational Context Modeling and Conversational Component Modeling. Context modeling focuses on how a system perceives the history of a chat, moving from simple sequential lists to graphical models that use Graph Convolutional Networks (GCNs) to represent utterances and interlocutors as nodes. Component modeling focuses on functional tasks like identifying who is speaking, what is being said (response selection/generation), and who is being addressed.

Taxonomy of Multi-Part Conversation Understanding, categorized into Context Modeling and Component Modeling.

Conversational Context Modeling

  • Information Flow: Modern approaches transition from sequential processing to graphical representations, mapping "reply-to" or "speaker-of" relationships.
  • Discourse Parsing: Techniques explicitly map the fine-grained structure of a conversation.
  • Self-supervision: Pre-trained models like MPC-BERT use specialized masks and auxiliary tasks (e.g., predicting the speaker or addressee) to learn the underlying structure of multi-party talk before fine-tuning.

Conversational Component Modeling

  • Speaker Modeling: Identifying speakers in unlabeled text and managing turn-taking.
  • Utterance Modeling: Response selection and generation, often utilizing "interlocutor-aware" contexts to maintain persona consistency.
  • Addressee Modeling: This includes explicit Addressee Recognition and implicit Dialogue Disentanglement, which involves figuring out which thread a message belongs to.

Key Insights and State-of-the-Art

The most significant shift in the field is the realization that MPCs are graphs, not sequences. Modeling the "reply-to" relationship as a directed edge allows models to bypass irrelevant interleaved messages. Recent state-of-the-art (SOTA) milestones include:

  • Graph-based models (like GSN and GCN-based approaches) consistently outperform sequential models in response generation and selection.
  • MPC-BERT demonstrated that pre-training on multi-party specific objectives significantly boosts performance on downstream tasks compared to general-purpose models like standard BERT.
  • Heterogeneity: Models like HeterMPC succeed by treating speakers and utterances as different types of nodes in a graph, rather than treating the chat as a homogeneous block of text.

Benchmarks and Datasets

Several key benchmarks drive evaluation in this field:

  • STAC: A corpus based on the game The Settlers of Catan, used for discourse parsing.
  • Ubuntu IRC & #Linux IRC: Large-scale technical chat logs essential for dialogue disentanglement.
  • DSTC8 (Track 2): A benchmark for multi-party response selection.
  • Literary Corpora: English and Chinese novels are used for speaker identification tasks where speakers are not explicitly tagged.

Future Directions

Despite rapid progress, several gaps remain for future research. There is a pressing need for Universal MPC Understanding—unified models that can handle the entire "Who/What/Whom" triple simultaneously rather than focusing on single tasks. Additionally, the field must move toward High-level Applications like multi-party summarization and machine reading comprehension. Finally, developing Low-resource Scenarios through zero-shot or few-shot learning is critical, as annotating MPC data remains exceptionally expensive and time-consuming.

Summary updated

Abstract

Multi-party conversations (MPCs) are a more practical and challenging scenario involving more than two interlocutors. This research topic has drawn significant attention from both academia and industry, and it is nowadays counted as one of the most promising research areas in the field of dialogue systems. In general, MPC algorithms aim at addressing the issues of Who says What to Whom, specifically, who speaks, say what, and address whom. The complicated interactions between interlocutors, between utterances, and between interlocutors and utterances develop many variant tasks of MPCs worth investigation. In this paper, we present a comprehensive survey of recent advances in text-based MPCs. In particular, we first summarize recent advances on the research of MPC context modeling including dialogue discourse parsing, dialogue flow modeling and self-supervised training for MPCs. Then we review the state-of-the-art models categorized by Who says What to Whom in MPCs. Finally, we highlight the challenges which are not yet well addressed in MPCs and present future research directions.
BibTeX
@inproceedings{gu2022who,
  author = {Jia-Chen Gu and Chongyang Tao and Zhen-Hua Ling},
  title = {{Who Says What to Whom: A Survey of Multi-Party Conversations}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2022},
  doi = {10.24963/ijcai.2022/768},
}
Figure