Hierarchical Structure for Multi Turn Dialogue Modeling

113 concepts100 papers1 proposal

Traditional natural language processing often treats text as a "flat" sequence—a continuous string of tokens where each word follows the next in a linear chain. While this works for sentences, it fails to capture the reality of human conversation. Dialogue is inherently multi-level: words form utterances, utterances form turns, and turns form complex, often non-linear threads.

Research in hierarchical dialogue modeling seeks to move beyond flat sequences by explicitly representing these nested relationships. By doing so, models can better resolve long-distance dependencies, maintain global coherence, and untangle the "spaghetti" of multi-party interactions.

The Information Bottleneck of Flat Sequences

When a model treats a conversation as a single long string, it struggles to distinguish between local word-level semantics and global discourse-level intent. As noted by Hao Tian et al. in Multi-view Response Selection for Human-Computer Conversation, the importance of overall dialogue structure grows as the number of turns increases. Without a hierarchical mechanism, models often lose track of context in long-form interactions.

To address this, Alessandro Sordoni et al. introduced the Hierarchical Recurrent Encoder-Decoder (HRED) in Hierarchical Neural Network Generative Models for Movie Dialogues. HRED uses two levels of recurrence: one to encode the sequence of words within an utterance, and another to encode the sequence of utterances within the dialogue. This allows the model to maintain a "dialogue state" that evolves turn-by-turn, rather than word-by-word, providing a more stable representation of the conversation's progress.

Capturing Non-Linear Propagation

In social media and multi-party chats, conversations are rarely linear. They branch into trees where a single post might trigger multiple independent sub-discussions. Treating these as flat timelines obscures the "topology" of the interaction.

The authors of Rumor Detection on Twitter with Tree-structured Recursive Neural Networks argue that the meaning of a reply is fundamentally dependent on its parent post. By using tree-structured networks, they capture how information propagates through nested replies. Similarly, Tian Bian et al. demonstrate in Rumor Detection on Social Media with Bi-Directional Graph Convolutional Networks that modeling both top-down propagation and bottom-up dispersion is critical for identifying rumors early.

This structural awareness is equally vital for detecting abuse. Célia Nouri et al. show in Graphically Speaking: Unmasking Abuse in Social Media with Conversation Insights that toxicity often hides in the context of a thread. A comment that appears neutral in isolation may become abusive when viewed as a specific reaction within a branching conversation tree.

Beyond simple nesting, hierarchical modeling allows systems to track what is being done in a conversation. João Sedoc argues in Hierarchical Methods for a Unified Approach to Discourse, Domain, and Style in Neural Conversational Models that modeling discourse acts (like questioning or agreeing) alongside topics and style is essential for coherence.

This is often achieved by linking concepts across turns. The authors of Semantic Representation for Dialogue Modeling use Abstract Meaning Representation (AMR) to build dialogue-level graphs. These graphs link identical concepts and coreferences across the conversation, allowing the model to "see" the underlying semantic roles—who is doing what to whom—even when those details are spread across multiple turns.

Adapting the Transformer

While Transformers have become the standard for NLP, their global attention mechanism can still suffer from a "linear bias." Bishal Santra proposed the HT-Encoder in Hierarchical Transformer for Task Oriented Dialog Systems, which uses specialized attention masks and dual-level positional encodings to force the model to respect utterance boundaries.

However, hierarchy comes with trade-offs. Rongyi Sun et al. point out in A non-hierarchical attention network with modality dropout for textual response generation in multimodal dialogue systems that compressing utterances into fixed vectors can create an information bottleneck, losing fine-grained word-level nuances. The challenge for modern research is finding the right balance: maintaining the high-resolution detail of tokens while leveraging the structural roadmap provided by the dialogue hierarchy.

History & citationVersion 2 · 2026-10-04

Current · Permanent link

  1. Version 2 (latest) · 2026-10-04 · 104 papers
  2. Version 1 · 2026-10-04 · 80 papers
Cite this direction

Latest version (a living page):

@misc{lacuna_hierarchical_dialogue_structure_hierarch_ed015cb0,
  title = {hierarchical dialogue structure (hierarchical discourse modeling)},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/hierarchical-dialogue-structure-hierarchical-discourse-modeling/rdr_ed015cb0b23c4f79a32484d9b201deac},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 2, frozen text):

@misc{lacuna_hierarchical_dialogue_structure_hierarch_ed015cb0_v2,
  title = {hierarchical dialogue structure (hierarchical discourse modeling)},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/hierarchical-dialogue-structure-hierarchical-discourse-modeling/rdr_ed015cb0b23c4f79a32484d9b201deac/v/2},
  note = {Version 2, frozen 2026-10-04. Accessed 2026-11-19},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake