Consider a piece of writing where every sentence transitions smoothly to the next using local connectives like however, therefore, or furthermore. On a sentence-by-sentence level, the text feels fluent. Yet, when you finish reading the entire document, you realize the argument has wandered aimlessly, completely failing to deliver a unified point.
This phenomenon highlights the local-global gap in discourse modeling. Traditional Natural Language Processing (NLP) systems often treat text as a linear sequence of sentences or evaluate discourse relations as isolated pairs. However, a document is not a flat chain of independent links; it is a nested hierarchy. Bridging this gap requires moving away from shallow, localized models toward global, tree-based representations of text structure.
The Failure of Local Link Independence
When models evaluate discourse relations strictly between immediate neighbors, they exhibit a form of structural short-sightedness. As Dongliang Liao et al. observe in Hierarchical Coherence Modeling for Document Quality Assessment, a text can maintain flawless local transitions while its global quality remains severely compromised by macro-structural disruptions.
This limitation stems from treating discourse links as independent events. In a linear sequence optimization framework, finding the right order for a set of sentences behaves like a Traveling Salesman Problem. As shown by Ernst Althaus et al. in Computing Locally Coherent Discourses, while linear paths can optimize local sentence-to-sentence transitions, they fundamentally fail to capture document-wide themes or overarching rhetorical structures.
Without a global blueprint, models suffer from a persistent gap between lexical-level precision and document-level structural completeness. Karthik Narasimhan et al. argue in Machine Comprehension with Discourse Relations that full document-level discourse trees are necessary to represent the complex, multi-layered relationships inherent in longer texts. Similarly, Yancui Li et al. demonstrate in Building Chinese Discourse Corpus with Connective-driven Dependency Tree Structure that local dependencies must be integrated into a unified global tree to truly represent a document's complete rhetorical flow.
The Mechanism of Hierarchical Trees
To capture global coherence, text must be modeled as a deep hierarchical tree where local arguments serve as leaves that roll up into broader structural nodes. Two prominent frameworks illustrate how this structural nesting works:
- Deep Rhetorical Structures: Instead of shallow, adjacent pairings, frameworks like Rhetorical Structure Theory (RST) map long-range dependencies across clauses, sentences, and entire paragraphs. Vanessa Wei Feng et al. demonstrate in The Impact of Deep Hierarchical Discourse Structures in the Evaluation of Text Coherence that deep hierarchical structures significantly outperform shallow, local relation models (such as the Penn Discourse TreeBank framework) when evaluating text coherence. Deep trees preserve the logical progression and overall unity of a text across long distances.
- Questions Under Discussion (QUD): This framework views a text as an implicit conversation where every sentence answers a latent question. Rather than treating these questions independently, Ashima Suvarna et al. explain in QUDSELECT: Selective Decoding for Questions Under Discussion Parsing that sub-questions must nest logically under broader parent questions to form a coherent global hierarchy. To enforce this, Yating Wu et al. emphasize in QUDeval: The Evaluation of Questions Under Discussion Discourse Parsing that models must satisfy global constraints to ensure that individual question-answer pairs form a logically consistent tree across the entire document.
This push for hierarchical organization is also vital for cross-linguistic analysis. For instance, Yuping Zhou et al. note in PDTB-style Discourse Annotation of Chinese Text that mapping long-range dependencies is essential to understand how individual arguments contribute to the overarching narrative of a document, especially in languages with heavy use of loose, comma-separated structures.
Enforcing Global Awareness in Neural Models
How do modern neural networks learn these global structures? Recent work shifts away from passive sequence modeling toward explicit structural enforcement and discourse-aware pre-training.
One effective strategy is to teach models what bad global structure looks like. In Corruption Is Not All Bad: Incorporating Discourse Structure into Pre-training via Corruption for Essay Scoring, Farjana Sultana Mim et al. introduce Discourse Corruption (DC). By intentionally shuffling paragraphs and training the model to identify these high-level structural disruptions, the system develops a robust understanding of paragraph-level transitions and macro-organization, proving far more effective than models trained merely on local sentence shuffling.
Linear Modeling (Short-sighted):
[Sentence 1] ──> [Sentence 2] ──> [Sentence 3] ──> [Sentence 4]
(Captures local transitions, misses overarching argument)
Hierarchical Tree Modeling (Global):
[Main Thesis]
/ \
[Paragraph A] [Paragraph B]
/ \ / \
[Sent 1] [Sent 2] [Sent 3] [Sent 4]
(Captures long-range dependencies and macro-structure)
In downstream applications, incorporating these trees yields measurable improvements. Wei Liu et al. address this in Discourse Relation-Enhanced Neural Coherence Modeling by designing a Fusion Transformer that utilizes position-aware attention and structural masking to explicitly integrate hierarchical discourse relations into neural networks.
Furthermore, probing analyses by Longyue Wang et al. in How Does Pretraining Improve Discourse-Aware Translation? reveal that a model's capacity to grasp these complex, long-range rhetorical structures directly correlates with its success in document-level challenges like machine translation. By shifting the paradigm from local link independence to global hierarchical trees, these approaches allow machines to move past surface-level fluency and comprehend the overarching logical blueprint of a text.
Key Foundational Work
Computing Locally Coherent Discourses (2004) — contributed 1 concept elements to this direction.
PDTB-style Discourse Annotation of Chinese Text (2012) — contributed 1 concept elements to this direction.