Most modern language models treat a conversation as a "flat" sequence—a single, long string of tokens. While this approach works for short snippets of text, it fundamentally misrepresents how humans actually communicate. Human conversation is not a linear stream; it is a nested hierarchy where words form utterances, utterances form turns, and turns aggregate into high-level discourse phases.
Researchers like Mihai Rotaru and Diane J. Litman argued early on in Exploiting Discourse Structure for Spoken Dialogue Performance Analysis that treating interactions as linear sequences ignores the hierarchical organization that gives user actions their meaning. To bridge this gap, a growing body of research focuses on architectures that explicitly model these multiple scales of interaction.
The Flat Sequence Fallacy
In a standard Transformer, every token can theoretically attend to every other token. However, in a multi-turn dialogue, the relationship between "Hello" in turn 1 and "Goodbye" in turn 20 is different from the relationship between "New" and "York" within a single sentence.
As Genta Indra Winata and colleagues point out in Hierarchical Attention for Dialogue Emotion Classification, flat models destroy the natural boundaries between speakers. This makes it difficult for a model to distinguish between local word-level dependencies and global, session-level flow. When a model treats a 50-turn transcript as one continuous block, the "signal" of the current intent often gets lost in the "noise" of the historical word matches.
Modeling the Three-Tiered Hierarchy
To capture the "deep structure" of conversation, researchers have moved toward multi-level architectures. These typically organize information into three distinct layers:
- The Token Level: Capturing local semantics and syntax within a single sentence.
- The Utterance Level: Representing the functional intent (or "Dialogue Act") of a specific turn.
- The Session Level: Tracking the global state, topic transitions, and long-term goals.
Maite Taboada demonstrated in Improving Translation through Contextual Information that tracking these layers allows systems to resolve "elliptical" phrases—short, ambiguous fragments like "twelve fifteen"—by looking at the current global phase of the dialogue (e.g., a time-setting phase) rather than just the preceding words.
Architectural Innovations for Nested Context
The shift from flat to hierarchical modeling has led to several key structural innovations:
- Hierarchical Transformers: Models like the HT-Encoder, proposed by Bishal Santra in Hierarchical Transformer for Task Oriented Dialog Systems, use specialized attention masks and dual-level positional encodings. This forces the model to process utterances as discrete units before integrating them into the broader dialogue context.
- Utterance Recurrence: Instead of standard token-level recurrence, DialogXL uses "utterance recurrence" to pass information across turns, effectively capturing inter-speaker influences and emotional trends that define conversational dynamics.
- Multi-View Representations: Hao Tian and the Baidu team developed a multi-view approach that simultaneously analyzes a conversation as a word stream and a structured sequence of turns, balancing fine-grained keyword matching with high-level discourse patterns.
Learning Discourse through Pretraining
If the architecture is hierarchical, the training objectives must be as well. Standard language modeling (predicting the next word) doesn't necessarily teach a model how a conversation progresses.
Shikib Mehri and Maxine Eskénazi introduced unsupervised objectives like Masked-Utterance Retrieval and Inconsistency Identification in Pretraining Methods for Dialog Context Representation Learning. Similarly, Xiaodong Gu developed DialogBERT, which uses Masked Utterance Regression to predict the representation of an entire missing turn. These tasks force the model to learn the "logic" of discourse—understanding not just what was said, but why it follows what came before.
Why Structure Matters
This hierarchical perspective is particularly vital in specialized domains. In medical consultations, Nan Wang and Fei Xia 0004 found that micro-level turns (like a doctor saying "Okay") only make sense when linked to macro-level clinical phases. Without hierarchical mapping, systems cannot track how specific interactions aggregate into medical outcomes.
By explicitly modeling the "hidden logic" of conversation—as seen in the DVAE-GNN model by Haifeng Wang—researchers are moving toward AI that doesn't just predict the next likely word, but understands its place within the complex, multi-layered dance of human interaction.