What makes a text readable? It isn't just a matter of choosing the right words or ensuring individual sentences are grammatically correct. A collection of perfectly formed sentences can still read like a jumbled "word salad" if the reader's attention is forced to jump erratically from one topic to another. As Mirella Lapata demonstrated in Probabilistic Text Structuring: Experiments with Sentence Ordering, maintaining topical focus across sentence boundaries relies heavily on how consistently a text references the same subjects.
To understand why, we must look at how humans process discourse. When we read, we construct an internal mental model populated by discourse entities—the people, places, things, and concepts driving the narrative. Local coherence is the measure of how smoothly a text guides our attention from one entity to the next across adjacent sentences.
The Theoretical Blueprint: Centering Theory
The formal foundation for tracking this attentional focus originates in Centering Theory, articulated by Grosz et al. in Providing a Unified Account of Definite Noun Phrases in Discourse. The core insight is that local discourse coherence is governed by a principle of focal consistency. Every sentence contains a ranked set of entities, with one primary entity acting as the "center" of attention.
When a text maintains the same central entity from one sentence to the next, it minimizes the cognitive load required for the reader to integrate new information. To signal this continuity, languages use specific syntactic cues. For instance, as Guindon et al. observed, speakers naturally transition from full noun phrases to pronouns as a specific topic progresses, re-introducing full nouns only when shifting to a new subdialogue. Conversely, abrupt topic changes—termed "Rough-Shifts" by Miltsakaki et al.—disrupt this flow and serve as strong quantitative indicators of poor writing quality.
The Computational Framework: The Entity Grid
While Centering Theory provided an elegant theoretical model, translating it into a scalable computational algorithm required a structural representation that machines could optimize. This breakthrough came with the introduction of the Entity Grid by Barzilay et al. in Modeling Local Coherence: An Entity-Based Approach.
An entity grid represents a text as a matrix where rows correspond to sequential sentences and columns represent unique discourse entities. Instead of merely tracking whether a word appears, each cell records the specific grammatical role of that entity within the sentence:
- S: Subject (the actor)
- O: Object (the recipient of the action)
- X: Oblique (involved in a prepositional phrase, etc.)
- -: Absence (the entity does not appear)
Why does this grammatical distinction matter? As Lapata et al. demonstrated in Automatic Evaluation of Text Coherence: Models and Representations, the structural patterns of how entities move across grammatical roles provide a highly reliable indicator of discourse flow, even if the actual meanings of the words are completely ignored.
Consider a transition where an entity is the Subject of sentence $n$ and remains the Subject of sentence $n+1$ ($S \rightarrow S$). This represents a highly stable, coherent flow where the narrative stays focused on the same actor. A transition where an object becomes a subject ($O \rightarrow S$) smoothly advances the narrative by shifting focus to a recipient of a previous action.
By calculating the probability distribution of these local transitions across the entire matrix, a system can quantify the structural smoothness of a text. As Karamanis et al. discovered, even the simplest metric—such as minimizing the frequency of sentences that fail to share any discourse entities with their immediate predecessor—provides a remarkably robust signal for readability.
Capturing Syntactic and Semantic Nuances
The basic entity grid assumes a rigid clausal structure, but human language is rarely that simple. To make entity tracking robust across diverse linguistic contexts, researchers have extended the framework along several dimensions:
- Syntactic Parallelism: Megumi Kameyama established in A Property-Sharing Constraint in Centering that maintaining the same grammatical function for an entity across sentences acts as a core constraint on discourse coherence, making structural parallelism an inherent mechanism for tracking central entities rather than an independent stylistic choice.
- Flexible Word Order: For languages like German where word order is highly flexible, traditional grammatical roles can be brittle. Cheung et al. resolved this by incorporating clausal "topological fields" into the entity matrix, making the model robust to parsing errors.
- Semantic Threads: A text might refer to "the doctor" in one sentence and "the hospital" in the next. To prevent the grid from fragmenting, Filippova et al. extended the model to group semantically related concepts into shared entity columns using external knowledge bases.
- Implicit and Omitted References: In pro-drop languages like Chinese, pronouns are frequently omitted when the discourse context is clear. Kong et al. showed that explicitly modeling these "zero pronouns" prevents the fragmentation of coreference chains, while Zhang et al. quantified how character-action continuity directly supports the predictability of these omissions.
Scaling to Neural Architectures and Text Generation
In the era of deep learning, the insights of entity-based coherence have transitioned from tabular matrices to neural network constraints and graph topologies. Traditional sequence models often suffer from a lack of global progression, producing text that feels repetitive or structurally disjointed because they lack a mechanism to explicitly track how the focus of a discussion shifts.
To address this, modern architectures explicitly embed entity tracking into their representations. For example, Yin et al. introduced a Sentence-Entity Graph that uses shared nouns to link sentences, filtering out the noise of fully connected models to dramatically improve sentence ordering. In data-to-text generation, Puduppully et al. designed an entity-centric neural framework that maintains a dynamic memory state for every entity, ensuring that references transition naturally (e.g., from full names to surnames) and remain factually consistent.
Furthermore, entity transition patterns are now used to evaluate and reward generation systems. Sharma et al. incorporated entity transition distributions directly into reinforcement learning rewards to optimize abstractive summarization. For evaluation, Jeon et al. developed neural local coherence models that generate interpretable focus maps based on entity transitions, while Steffen Eger operationalized Centering Theory using contextualized embeddings in DiscoScore: Evaluating Text Generation with BERT and Discourse Coherence, moving evaluation beyond surface-level token similarity to capture true structural coherence.
Key Foundational Work
Control of Inference: Role of Some Aspects of Discourse Structure-Centering (1981) — contributed 4 concept elements to this direction.