Standard language models process text as a linear sequence of tokens. While this works for prose, it struggles with the multi-dimensional nature of human conversation. In a dialogue—especially one involving multiple participants—information does not just flow from left to right. It flows between specific speakers, across intertwined threads, and through complex social and rhetorical relationships.
To capture this, researchers have moved toward representing conversations as heterogeneous graphs. By treating speakers, utterances, and semantic concepts as distinct types of nodes connected by specialized edges, these models can reason about the "who, what, and whom" of a conversation in a way that flat text sequences cannot.
The Limitation of Linear Sequences
In a multi-party chat or a transcript of a meeting, a single linear stream of text hides the underlying structure. An utterance might respond to a comment made five turns ago, or two people might be having a side conversation while a third person presents.
When a model like a standard Transformer processes this linearly, it relies on self-attention to "discover" these links. However, as Jia-Chen Gu, Chongyang Tao, and Zhen-Hua Ling argue in Who Says What to Whom: A Survey of Multi-Party Conversations, treating dialogue history as a uniform block of text ignores the distinct roles of participants. A linear model often fails to distinguish between the content of a message and the persona of the speaker, leading to confusion in complex social dynamics.
Dialogue as a Relational Topology
The core intuition behind this research direction is to replace the sequence with a graph. In a Heterogeneous Graph Network, the model is given a specific "map" of the conversation's structure:
- Diverse Node Types: Instead of every token being equal, the graph contains different categories of nodes. For example, Dianbo Sui and Kang Liu in Graph-Based Knowledge Integration for Question Answering over Dialogue use nodes to represent both entities and speakers, allowing the model to explicitly track social relationships.
- Relational Edges: Edges are not just connections; they are typed. An edge might represent "Speaker A said Utterance 1," "Utterance 2 responds to Utterance 1," or "Utterance 3 mentions Entity X."
This structure allows models like HeterMPC, developed by Jia-Chen Gu and colleagues in HeterMPC: A Heterogeneous Graph Neural Network for Response Generation in Multi-Party Conversations, to capture the "who-said-what-to-whom" dynamics. By separating utterance nodes from speaker nodes, the model can learn speaker-specific behavioral patterns and track how information propagates through a group.
Capturing Discourse and Intent
Beyond just tracking who is speaking, heterogeneous graphs allow models to understand the rhetorical purpose of a turn. In Dialogue Discourse-Aware Graph Convolutional Networks for Abstractive Meeting Summarization, Xiachong Feng and Bing Qin introduce DDAMS, which uses edges to represent discourse relations like "Contrast" or "Question-Answer."
This is particularly effective for tasks like Dialogue Disentanglement—the process of separating intertwined conversation threads. Fei Li and Hao Fei demonstrate in Revisiting Conversation Discourse for Dialogue Disentanglement that by modeling speaker identity, user mentions, and temporal distance as distinct edge types, a graph model can "unweave" a messy chat room log more accurately than even large-scale sequential models like GPT-3.5.
Bridging Local Turns and Global Context
One of the most powerful aspects of this approach is its ability to integrate different levels of granularity. A conversation has local dynamics (the immediate back-and-forth) and global themes (the overall topic).
Models like SumAggGIN, proposed by Dongming Sheng and Ying Shen in Summarize before Aggregate: A Global-to-local Heterogeneous Graph Inference Network for Conversational Emotion Recognition, use heterogeneous graphs to bridge these levels. They define nodes for both broad sequence segments and specific key phrases. By weighting the edges between them, the model can see how specific vocabulary choices across the entire conversation contribute to the emotional state of a single turn.
Similarly, Xiachong Feng and Ting Liu show in Incorporating Commonsense Knowledge into Abstractive Dialogue Summarization via Heterogeneous Graph Networks that external knowledge (like ConceptNet) can be added as a third node type. This allows the model to "fill in the blanks" of what is left unsaid, using the graph to link spoken utterances to the underlying commonsense concepts they imply.
Why It Works: Relational Inductive Bias
The success of heterogeneous graphs in dialogue stems from relational inductive bias. While a Transformer can learn these relationships, a graph-based model is told they exist. By explicitly defining the architecture around the way humans actually converse—referencing past turns, addressing specific people, and following discourse logic—these models achieve better results with less data and greater resistance to noise.
Whether it is predicting financial risk from conference calls in DialogueGAT by Yunxin Sang or synthesizing emotional speech in ECSS by Rui Liu, the shift from "text as a line" to "dialogue as a graph" provides the structural clarity needed to master human interaction.