A text can be grammatically perfect and locally fluent while remaining a complete failure as a document. Consider a "word salad" of sentences where each sentence shares a keyword with the next: the transitions are smooth, but the overall message is non-existent. This phenomenon exposes a fundamental gap in natural language processing: the difference between local coherence (sentence-to-sentence flow) and global coherence (the overarching logical structure of a document).
Research in this area seeks to move beyond "short-sighted" models that only look at adjacent clauses, instead developing frameworks that capture the "skeleton" or "rhetorical map" of an entire text.
The Limits of Local Flow
Early breakthroughs in discourse focused on how entities (people, places, or things) move through a text. In their seminal work, Regina Barzilay and Mirella Lapata introduced the Entity Grid, which tracks the grammatical roles of entities across adjacent sentences. If a subject in sentence A remains the subject in sentence B, the text is likely coherent.
However, as noted in Neural Automated Essay Scoring and Coherence Modeling for Adversarially Crafted Input, these models are vulnerable to adversarial "word salads." A document can maintain local entity transitions while lacking a global logical structure. This is particularly problematic in automated essay scoring, where a student might write a series of related but disorganized thoughts that fail to form a cohesive argument.
From Linear Chains to Hierarchical Trees
To capture the "big picture," researchers have turned to Rhetorical Structure Theory (RST). Instead of a linear chain of sentences, RST views a document as a hierarchical tree where small units (clauses) combine to form larger functional blocks (arguments, evidence, or backgrounds).
Daniel Marcu pioneered this shift with A Decision-Based Approach to Rhetorical Parsing, treating the discovery of this tree structure as a series of supervised machine learning decisions. By understanding how a "claim" in the first paragraph is supported by "evidence" in the third, models can evaluate the global organizational principles that link disparate parts of a text into a unified whole.
This structural perspective is critical for evaluation. Shafiq Joty and colleagues demonstrated in DiscoTK and related work that comparing the rhetorical trees of a machine-translated document against a human reference provides a much more accurate measure of quality than traditional word-matching metrics like BLEU.
Modeling the Global Skeleton
Modern neural approaches attempt to integrate these structural insights directly into model architectures. Dongliang Liao introduced HierCoh in Hierarchical Coherence Modeling for Document Quality Assessment, which uses "Max-Coherence Pooling" to capture high-level discourse patterns. Rather than just checking if sentence A relates to sentence B, these models look for a "global rhythm" or canonical template—for instance, ensuring a scientific abstract moves from "Background" to "Results" rather than looping back to the introduction at the end.
Other researchers, such as Wei Liu 0145, have moved toward graph-based representations. In Modeling Structural Similarities between Documents for Coherence Assessment with Graph Convolutional Networks, documents are treated as heterogeneous graphs where nodes are connected by shared structural subgraphs. This allows the model to explicitly see the topological connectivity of a document, identifying when the "logical flow" is interrupted by a misplaced paragraph or an abrupt topic shift (often called a "Rough-Shift," as explored by Eleni Miltsakaki in The Role of Centering Theory's Rough-Shift).
Why Global Coherence Matters
The push for global reasoning spans multiple subfields of NLP:
- Text Simplification: Wei Xu and Mounica Maddela argue in their work on LENS that sentence-level metrics fail to capture whether a simplified text remains a coherent narrative.
- Media Bias: Gisela Vallejo and Lea Frermann point out in Connecting the Dots in News Analysis that bias is rarely found in a single sentence; it emerges from how a story is framed across an entire document.
- Text Segmentation: Identifying where a chapter ends in a novel requires balancing local semantic shifts with a "global rhythm" constraint, as shown in Chapter Captor.
Ultimately, the goal is to move from models that are merely fluent to models that are coherent—capable of reasoning across time, documents, and complex rhetorical structures to understand not just what is being said, but how the entire message hangs together.