Modeling Conversations as Heterogeneous Graphs

27 concepts23 papers2 proposals

Standard language models treat text as a linear sequence of tokens. While this paradigm works exceptionally well for documents, it struggles with the inherent structure of human dialogue. A conversation is not a flat sequence; it is a dynamic, multi-party web of interactions. As Gu et al. note in their survey of multi-party conversations, understanding dialogue requires tracking "who says what to whom."

When we flatten a transcript into a single block of text, we force the model to use its attention mechanism to reconstruct basic structural facts, such as speaker identity, turn sequence, and topic transitions. To bypass this sequential bottleneck, a growing body of research models conversations as heterogeneous graphs. By representing different conversational elements as distinct node types and connecting them with specialized, typed edges, these architectures explicitly encode the non-linear dependencies of human discourse.


Disentangling Speakers from Utterances

In a multi-speaker conversation, a participant's current emotional state or intent is heavily influenced by their own past statements (speaker consistency) and the statements of others (contextual influence). In a standard sequence model, a speaker's identity is typically reduced to a text prefix (e.g., "Alice: [text]"). To maintain a persistent representation of Alice across twenty turns of intervening banter from other participants, the model must propagate information through a mountain of sequential noise.

Sequential Flattening (Information gets lost over long distances):
[Alice: Hello] -> [Bob: Hi] -> [Charlie: Hey] -> [Alice: How are you?]

Heterogeneous Graph (Direct structural routing):
(Speaker: Alice) ------> (Utterance 1: "Hello")
       |
       +----------------> (Utterance 4: "How are you?")

Heterogeneous graphs solve this by treating speakers and utterances as fundamentally different entities. In the ConGCN framework introduced by Zhang et al., utterances and speakers are mapped to separate node types. This allows the network to route information along two distinct pathways simultaneously: context-sensitive pathways (utterance-to-utterance) and speaker-sensitive pathways (speaker-to-utterance).

By decoupling the participant from the message text, the graph structure creates a direct line of communication between a speaker's historical turns. The authors demonstrate that this architecture significantly improves emotion detection by preserving individual personality traits across long contexts.

This structural separation proves vital across several dialogue tasks:

  • Response Generation: Gu et al. developed HeterMPC, which utilizes distinct speaker and utterance nodes to capture intricate social structures and dialogue history in multi-party settings, outperforming uniform sequence models.
  • Intent Classification: Pyarelal et al. show that explicitly linking speaker nodes to their respective utterances creates "speaker-enriched" representations that help classify rare conversational intents in complex, many-participant datasets.
  • Relation Extraction: Chen et al. use a heterogeneous graph attention network to link speakers, utterances, and entity types, allowing models to extract deep relational knowledge (such as family ties or professional roles) buried across disjointed conversational turns.
  • Domain-Specific Modeling: In the financial sector, Sang et al. designed DialogueGAT to predict stock volatility from earnings conference calls by explicitly modeling the relationships between corporate executives, analysts, and their specific statements.

Multi-Grained Modeling: From Tokens to Discourse

Conversations operate at multiple scales simultaneously. At a micro level, words govern immediate syntactic dependencies and local topic transitions. At a macro level, utterances govern the global flow of the conversation and high-level discourse structures.

Heterogeneous graphs naturally accommodate this multi-grained reality by stacking or nesting graph branches. For example, the Multi-Grained Conversational Graph Network (MCGN), proposed by Tu et al., operates a dual-branch architecture. One branch maps fine-grained word-to-word topic transitions, while the other maps coarse-grained utterance-to-utterance conversational flow. This allows a retrieval-based chatbot to evaluate both local semantic shifts and global structural coherence when selecting a response.

[ Dialogue Level Node ]
         |
   +-----+-----+
   |           |
[Turn 1]    [Turn 2]  <-- Utterance Level (Global Flow)
   |           |
 [Words]     [Words]  <-- Token Level (Local Semantics)

Similarly, the DMIN architecture designed by Huang et al. uses a dual-stream graph convolutional network to process token-level syntactic dependencies alongside utterance-level speaker reply patterns for sentiment quadruple analysis.

By organizing dialogue hierarchically across word, sentence, and dialogue levels—as seen in the speech synthesis work of Liu et al. and the HiDialog framework by Liu et al.—models can bridge remote interactions and maintain structural coherence over long exchanges without requiring expensive, long-context pre-training.


Anchoring Intents, History, and Abstract Knowledge

One of the most powerful features of the heterogeneous graph paradigm is its ability to blend raw text with abstract semantic concepts and external knowledge within the same topological space. Instead of forcing a model to learn implicit relationships through data exposure, researchers can explicitly anchor these concepts as nodes.

Intent Anchors and User History

In Dialogue Act Recognition (DAR), expressions of intent are often highly personalized and informal. Wang et al. addressed this by creating the Heterogeneous User History (HUH) graph framework. The graph contains nodes for current utterances, but also nodes representing abstract intent categories and historical user patterns. By linking current turns to classification labels through shared historical context, the graph acts as a structural denoising mechanism, making it easier to detect subtle intents like "Negative Feedback."

Goal Planning and Recommendations

In conversational recommender systems, a bot must seamlessly transition from casual chitchat to information gathering, and finally to a product recommendation. Zhang et al. introduced GIGF (Goal Interaction Graph Planning), which integrates conversational goal types and specific entities into a single heterogeneous graph. This structure allows the system to discover cross-level correlations, recognizing exactly how a concrete entity (e.g., a movie director) can serve as a bridge between different abstract dialogue stages.

Mental States and Modalities

The flexibility of heterogeneous graphs extends to modeling internal psychological states and multi-modal inputs:

  • Belief-Desire Modeling: Xu et al. integrated the Belief-Desire Theory of Emotion into a heterogeneous graph. By treating utterances, speakers, and extracted events as distinct nodes, the model simulates how a speaker's internal desires interact with conversational events to trigger fine-grained emotions.
  • Multi-Modal Synthesis: For conversational speech synthesis, the ECSS framework by Liu et al. constructs a graph containing text, audio features, speaker identity, and emotional attributes. A graph transformer processes these diverse nodes to capture non-linear "emotion flow," generating more expressive and contextually appropriate speech.
  • Linguistic Arguments: Lee et al. developed TUCORE-GCN, which inserts relational arguments (like subjects and objects) as distinct nodes linked to their source utterances, ensuring the model preserves core semantic relationships across multi-turn conversations.
  • Knowledge Integration: For question answering over dialogues, Liu et al. built relational graphs where words are nodes, and edges explicitly encode co-reference and semantic relations. This structural grounding significantly enhances the model's robustness against adversarial attacks.

By moving away from flat text sequences and embracing heterogeneous graphs, these diverse frameworks demonstrate a unified truth: when a model's internal architecture mirrors the multi-layered, multi-party structure of human conversation, it understands not just what is being said, but the complex social and semantic context that gives it meaning.

Key Foundational Work

Integrating User History into Heterogeneous Graph for Dialogue Act Recognition (2020) — contributed 2 concept elements to this direction.
Modeling both Context- and Speaker-Sensitive Dependence for Emotion Detection in Multi-speaker Conversations (2019) — contributed 1 concept elements to this direction.
Dialogue Relation Extraction with Document-level Heterogeneous Graph Attention Networks (2020) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_heterogeneous_dialogue_graph_b48794a7,
  title = {heterogeneous dialogue graph},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/heterogeneous-dialogue-graph/rdr_b48794a7f34d41cd92739606139d0a20},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 1, frozen text):

@misc{lacuna_heterogeneous_dialogue_graph_b48794a7_v1,
  title = {heterogeneous dialogue graph},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/heterogeneous-dialogue-graph/rdr_b48794a7f34d41cd92739606139d0a20/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake