Robust Participant Tracking for Dialogue Understanding

This direction is split into other directions: speaker attribution error. Its last version stays readable and citable here.

Version 1 of 1107 papers140 conceptsfrozen 2026-10-04

While modern Large Language Models (LLMs) can generate remarkably fluent responses, they frequently suffer from what researchers call an "identity crisis" in multi-party conversations. They can capture the "what" of a discussion—the topics, the emotions, and the facts—but they often fail at the "who." This failure to maintain a stable internal map of participants leads to speaker-action misattribution, where a model correctly identifies an event but assigns it to the wrong person.

The Logical Gap Between Fluency and Attribution

Linguistic fluency does not guarantee a robust representation of conversational logic. As Shuaijie She and colleagues demonstrate in Exploring the Dialogue Comprehension Ability of Large Language Models, even state-of-the-art models like ChatGPT fail to be factually consistent in roughly 16% of summaries, primarily due to swapping subjects and objects or omitting participants entirely.

This "who did what to whom" problem is particularly acute in multi-turn dialogues. Models often struggle with subject-object swaps, where the roles of the speaker and the listener are inverted. For instance, in a medical context, a model might correctly extract a symptom but attribute it to the doctor instead of the patient. Yuan Xia and the team behind the SAFE framework argue that even speaker-aware pre-training is insufficient; models require explicit mechanisms to model the complex relationships between different parts of a conversation to avoid these critical failure modes.

Why Participant Tracking Fails

The difficulty of tracking participants stems from several conversational dynamics:

  • Coreference and Pronouns: Conversations are dense with pronouns and informal language. Xiangru Tang notes in CONFIT that nearly half of factual errors in dialogue summarization involve misattributing information, often because models lose the thread of who a pronoun refers to across multiple turns.
  • Distractor Entities: In tasks like quote attribution, the presence of other names in the surrounding text often misleads models. Wenjie Zhong and Hiroya Takamura found that models struggle to distinguish the actual speaker from other entities mentioned in the context, even when the speaker's name is explicitly nearby.
  • Persona Leakage: In retrieval-based chatbots, a lack of clear differentiation between the bot's persona and the user's persona can lead to "persona leakage." Jia-Chen Gu shows in Partner Matters! that if the two personas are not processed distinctly, the system may incorrectly attribute the user's traits or experiences to itself.

The Propagation of Diarization Errors

In real-world applications, the problem begins before the text reaches the NLP model. Automatic Speech Recognition (ASR) systems often struggle with speaker diarization—the process of determining "who spoke when."

Research by Nikolaos Flemotomos in the domain of psychotherapy evaluation reveals a surprising pattern: errors in speaker diarization have a more significant negative impact on behavioral coding accuracy than actual word-level transcription errors. If a system misattributes a therapist's empathetic statement to the patient, the entire quality metric for that session becomes invalid. This challenge is echoed in the work of Abhinav Ramesh Kashyap, whose MEDSAGE framework uses LLMs to generate synthetic "noisy" dialogues to help models become more robust to these inevitable diarization failures.

Architectural Solutions for Speaker Awareness

To solve these issues, researchers are moving beyond simple text concatenation toward "speaker-aware" architectures:

  1. Explicit Structural Markers: Deyi Xiong and Jun Quan demonstrate that simple utterance tagging—explicitly labeling each turn with a speaker ID—significantly improves dialogue state tracking by helping the model distinguish between user requirements and system prompts.
  2. Heterogeneous Graph Networks: Models like the Dialogue Heterogeneous Graph Network (D-HGN), proposed by Xiachong Feng, treat speakers, utterances, and concepts as distinct nodes in a graph. This allows the model to explicitly learn the "subtext" and relational dynamics of the conversation.
  3. Post-Correction Pipelines: Rather than trying to get it right the first time, some researchers use a second pass. Dongyub Lee and colleagues developed a speaker-focused post-correction pipeline that specifically scans generated summaries for swapped, added, or omitted names, significantly improving factual accuracy.

High-Stakes Implications

The necessity of robust participant tracking is most evident in specialized domains. In legal analysis, Elliott Ash points out that a lack of speaker attribution prevents models from distinguishing between a judge's normative rulings and a witness's reported testimony. In mental health, Bichen Wang notes that suicide risk detection systems often fail because they cannot differentiate between a user's self-disclosure and their discussion of someone else's behavior.

Ultimately, the goal of this research direction is to move AI from being a "fluent listener" to a "socially aware participant" that understands the conversation not just as a sequence of words, but as a structured interaction between distinct identities.

Papers in this version

Showing 60 of 107 papers.

History & citationVersion 1 · 2026-10-04

Split into other directions · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_robust_participant_tracking_for_dialogue_a2a9ed58,
  title = {Robust Participant Tracking for Dialogue Understanding},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/robust-participant-tracking-for-dialogue-understanding/rdr_a2a9ed58ebd846eba441f35d523db5ec},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_robust_participant_tracking_for_dialogue_a2a9ed58_v1,
  title = {Robust Participant Tracking for Dialogue Understanding},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/robust-participant-tracking-for-dialogue-understanding/rdr_a2a9ed58ebd846eba441f35d523db5ec/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}