When we read a well-written paragraph, we don't just process a list of facts; we follow a thread. If a writer suddenly switches topics or refers to a "he" without a clear antecedent, the text feels disjointed. This "hanging together" of sentences is known as local coherence.
For decades, researchers have argued that the secret to local coherence lies not in the words themselves, but in how a text manages our attention. By tracking specific entities—the people, places, and things mentioned—and observing how they transition between grammatical roles, we can mathematically model the "flow" of a document.
The Cognitive Foundation: Centering Theory
The modern understanding of entity-based coherence began with Centering Theory. In their foundational work, Aravind K. Joshi and Scott Weinstein proposed that discourse structure is a mechanism for controlling "inferential load" Control of Inference: Role of Some Aspects of Discourse Structure-Centering.
The intuition is simple: at any given moment in a conversation, there is a single entity that is the "center" of attention. Barbara J. Grosz, Joshi, and Weinstein formalized this into two structures Providing a Unified Account of Definite Noun Phrases in Discourse:
- The Backward-Looking Center ($C_b$): The entity that links the current sentence to the previous one.
- The Forward-Looking Centers ($C_f$): A ranked list of entities in the current sentence that are candidates to become the focus of the next sentence.
Coherence is maintained when the $C_b$ remains stable across sentences. When a speaker uses a pronoun, they signal that the focus is continuing; when they use a full noun phrase, they may be signaling a shift The Structure of User-Adviser Dialogues: Is there Method in their Madness?. As Susan Brennan and colleagues demonstrated, this framework provides a powerful algorithm for resolving pronouns by simply tracking which entity is currently "centered" A Centering Approach to Pronouns.
From Theory to Matrix: The Entity Grid
While Centering Theory provided the linguistic rules, Regina Barzilay and Mirella Lapata turned it into a scalable machine learning tool with the Entity Grid Modeling Local Coherence: An Entity-Based Approach.
Instead of just looking for word overlap, the Entity Grid represents a document as a matrix where rows are sentences and columns are entities. Each cell records the grammatical role of that entity: is it the Subject (S), the Object (O), or something else (X)?
| Entity: "The Dog" | Entity: "The Ball" | Entity: "The Park" | |
|---|---|---|---|
| S1 | S | O | X |
| S2 | S | - | O |
| S3 | S | - | - |
By looking at this grid, we can see a "Subject-to-Subject" transition for "The Dog." Barzilay and Lapata showed that coherent, human-written texts follow predictable patterns in these transitions—for example, the primary subject of a story tends to stay in the subject position across adjacent sentences. This approach allows models to distinguish a real news article from a randomly shuffled version of the same sentences Probabilistic Text Structuring: Experiments with Sentence Ordering.
Why Grammatical Roles Matter
A key insight of this research direction is that not all mentions are equal. Megumi Kameyama argued that "property-sharing"—specifically maintaining the same grammatical role—is a core constraint on coherence A Property-Sharing Constraint in Centering.
If an entity is the Subject of sentence A and the Subject of sentence B, the transition is "smooth." If it jumps from Object in sentence A to Subject in sentence B, the reader must perform more cognitive work to re-center their attention. Rebecca J. Passonneau found that this role persistence is the single best predictor of whether a speaker will use a simple pronoun like "it" versus a more complex demonstrative like "that" Getting at Discourse Referents.
Modern Applications: Beyond "Toy" Tasks
Today, entity-based modeling has moved beyond simple sentence shuffling into the heart of modern NLP:
- Sentence Ordering: Yongjing Yin and colleagues use Sentence-Entity Graphs to link sentences via shared nouns, filtering out the noise of traditional models to reconstruct logical narratives Graph-based Neural Sentence Ordering.
- Evaluation Metrics: Standard metrics like BLEU often miss the "forest for the trees." New metrics like DiscoScore Michael Strube combine BERT embeddings with Centering Theory to ensure that AI-generated summaries don't just contain the right words, but actually flow logically DiscoScore: Evaluating Text Generation with BERT and Discourse Coherence.
- Automated Scoring: In education, tracking "Rough-Shifts"—abrupt entity changes—has become a powerful way for systems like e-rater to mimic human judgments of essay quality Eleni Miltsakaki and Karen Kukich The Role of Centering Theory's Rough-Shift in the Teaching and Evaluation of Writing Skills.
By treating a document as a series of entity transitions rather than a bag of words, this research direction provides a formal bridge between the way humans think and the way machines process language.