Standard language models typically view dialogue as a linear sequence of tokens—a chronological chain where turn $A$ leads to turn $B$. However, human conversation is rarely a straight line. In multi-party chats, meetings, and even complex two-person debates, dialogue is "criss-crossed" and interleaved. Multiple topics often run in parallel, and a speaker might respond to a comment made ten turns ago while ignoring the immediate predecessor.
To capture this "messy" reality, researchers have moved away from sequential modeling toward representing conversations as structured graphs. By treating utterances as nodes and their relationships as edges, models can navigate the non-linear flow of human interaction.
The Failure of the Linear Assumption
Traditional models like BERT or early Transformers process text as a flat sequence. As Zhuosheng Zhang and Hai Zhao argue, this approach hits a wall in multi-turn dialogues where speaker roles and temporal flows are complex. In a group chat, for instance, three people might be discussing lunch while two others troubleshoot a server error. A linear model sees these as a single, incoherent stream of text.
Stergos D. Afantenos and Jiaqi Li demonstrate that multi-party dialogues are better represented as Directed Acyclic Graphs (DAGs). In this view, an utterance is not just "after" the previous one; it "attaches" to specific preceding units. This structural shift allows models to "disentangle" interleaved threads, a concept Pascal Poupart and Wenchao Du used to map dependencies in asynchronous customer service logs.
Mapping the Relational Web
If a conversation is a graph, what defines the edges? Research suggests several layers of connectivity:
- Discourse Relations: Xiachong Feng and Bing Qin introduced DDAMS, which uses explicit discourse labels like "Question-Answer" or "Contrast" to link utterances. This allows a model to bridge long-range dependencies, such as connecting a resolution to a question asked much earlier in a meeting.
- Speaker and Persona: Dan Su and Jingbei Li use DialogueGCN to model the "vibe" of a conversation, capturing how speakers influence each other (inter-speaker influence) and maintain their own style (intra-speaker inertia).
- Multi-Relational Attention: Yifu Guo proposes DialogGraph-LLM, which uses edges to capture temporal sequence, speaker identity, and semantic similarity simultaneously. This multi-layered approach helps Large Language Models (LLMs) recognize intent by understanding the "Who-What-Whom" dynamics identified by Jia-Chen Gu and Zhen-Hua Ling.
Reasoning Across Distant Turns
The primary advantage of a graph structure is its ability to bypass "noise." In a long conversation, the most relevant context for the current turn might be 20 turns back. In a linear model, this information is often diluted by the intervening text.
Hung Le and Steven Hoi developed the Reasoning Paths (PDC) framework to solve this. By treating dialogue as a semantic graph, the model can "jump" across the graph to retrieve critical cues from the distant past. Similarly, Wenguan Wang and Song‐Chun Zhu frame visual dialogue as inference in a latent graph, allowing the system to recover hidden structures and link current queries to entities mentioned much earlier.
This "structural memory" is particularly vital in specialized domains. Yu Zhao and Baotian Hu use knowledge graphs to bridge non-linear relationships in medical dialogues, ensuring that a doctor's recommendation remains grounded in symptoms mentioned at the start of the consultation.
From Chains to Trees and DAGs
The evolution of this field can be seen as a progression in topological complexity:
- Trees: Yao Dou and Yejin Choi use branching trees to model diverse conversational trajectories in MultiTalk, while Wendi Li uses trees to capture causal links between entities for recommendations.
- Fully Connected Graphs: Shafiq Joty and Preslav Nakov experimented with models where every comment can influence every other comment, capturing non-sequential dependencies in community forums.
- Dynamic Causal Graphs: Most recently, Shuchang Pan and colleagues have introduced Graph-of-Thoughts (GoT) for real-time voice assistants, allowing the system to reason about high-level intents and future interaction strategies as a dynamic, evolving web.
By moving from the "chain" to the "graph," these researchers have provided a way for AI to finally handle the non-linear, multi-threaded, and often chaotic nature of how humans actually talk.