The Coreference Resolution Bottleneck in Document Understanding

502 concepts395 papers3 proposals

When a human reads a narrative, they effortlessly track the cast of characters. If a text introduces "Apple Inc.," and subsequently refers to it as "the company," "the tech giant," or simply "it," our mental model maintains a single, continuous file for that entity. This phenomenon is known as entity persistence.

For natural language processing models, however, maintaining this file across sentence boundaries remains a profound challenge. While modern architectures excel at extracting facts from isolated sentences, their understanding degrades rapidly when forced to synthesize information across a whole document. This breakdown stems from the coreference resolution gap: the inability of models to reliably link pronouns, aliases, and implicit descriptions back to their canonical referents.

Why Standard Attention Fails at Identity

It is tempting to assume that the self-attention mechanism inherent to Transformers natively solves coreference. After all, attention allows tokens to look back at any previous position in a text. Why can't a model simply attend to the antecedent?

The issue lies in the difference between semantic similarity and discrete identity. Self-attention computes a soft, continuous distribution based on contextual relevance. If a sentence contains the pronoun "it," attention might distribute weight across several nouns that fit grammatically or topically. However, coreference requires a hard, structural chain of identity.

As Bhuwan Dhingra et al. argue in Neural Models for Reasoning over Multiple Mentions Using Coreference, general-purpose attention mechanisms do not inherently model these explicit chains. Without an explicit bias or architectural structure that enforces entity persistence, models struggle to pass information down a specific reference chain, forcing them to guess relationships based on local context clues rather than underlying logic.

How the Coreference Gap Sabotages Downstream Tasks

When a model cannot track entity identity, the failure propagates through every high-level text processing task, acting as a severe performance bottleneck.

Information and Event Extraction

To extract a complete picture of an event, a system must aggregate details spread across multiple paragraphs. Ruihong Huang et al. demonstrate this vulnerability in Peeling Back the Layers: Detecting Event Role Fillers in Secondary Contexts, showing that event extraction systems frequently fail to fill template roles because they cannot resolve pronominal references across sentence boundaries.

This problem is equally acute in specialized domains. In biomedical texts, Emma Strubell et al. show in Simultaneously Self-Attending to All Mentions for Full-Abstract Biological Relation Extraction that document-level relation extraction is fundamentally limited by how well a model resolves complex, distributed mentions of biological entities. Similarly, in financial analysis, Xianchao Wu notes in Event-Driven Learning of Systematic Behaviours in Stock Markets that financial news heavily relies on indirect references; without resolving these to specific entities, extracted actions remain orphaned and contextless, rendering them useless for market prediction.

Fact Verification and Summarization

Automated fact-checking requires verifying claims against a corpus of evidence. Andreas Hanselowski et al. observe in UKP-Athene: Multi-Sentence Textual Entailment for Claim Verification that verification models routinely fail when evidence is fragmented across sentences and relies on unresolved pronouns or generic descriptors.

The inverse problem occurs during text generation and summarization. When a model extracts a sentence to rewrite or compress it, it often carries over pronouns or shorthand references. As Guangsheng Bao et al. point out in A General Contextualized Rewriting Framework for Text Summarization, if the rewriting phase lacks access to the full coreference context of the source text, the generated output becomes ambiguous or factually incorrect because the pronouns lose their anchors.

Opinion and Sentiment Attribution

In sentiment analysis, identifying what opinion is expressed is trivial compared to identifying who holds it. Eric Breck et al. highlight in Cornell System Description for the NTCIR-6 Opinion Task that fine-grained opinion mining is deeply hindered by coreference failures. If a speaker is introduced in sentence one, and expresses opinions via pronouns in sentences two through five, a model without robust coreference tracking will fail to attribute those sentiments to the correct holder.

The Cross-Document Multiplier

The coreference gap compounds when models must reason across entirely different documents. In retrieval-heavy tasks, a model might find all relevant documents but still fail to synthesize them.

Marten Postma et al. expose this in SemEval-2018 Task 5: Counting Events and Participants in the Long Tail, revealing a stark gap between a system's ability to retrieve relevant texts and its ability to accurately count unique entities or events across them. Because models struggle with cross-document coreference, they cannot determine whether two separate news reports are describing the exact same individual or two distinct entities.

In context-poor environments, such as social media comments, the antecedent might not even exist in the same text block. To resolve this, Xinyu Hua et al. demonstrate in XREF: Entity Linking for Chinese News Comments with Supplementary Article Reference that models must explicitly look outside the immediate text—leveraging parent articles as supplementary reference fields—to map ambiguous nicknames and pronouns to their true global entities.

Bridging the Gap: Explicit Structural Geometry

To overcome this bottleneck, researchers are moving away from the hope that standard language models will "implicitly" learn coreference. Instead, they are designing architectures that explicitly inject coreference geometry into the model's representations.

One prominent strategy is to shift from token-level modeling to explicit mention-mention graphs. Dhruv Agarwal et al. propose this paradigm in Entity Linking via Explicit Mention-Mention Coreference Modeling. By training dual-encoders to construct directed trees across mentions across different documents, they force the model to leverage the diverse linguistic contexts of all mentions simultaneously, significantly improving disambiguation performance.

[Mention 1: "Apple"] <=======> [Mention 2: "The company"]
       \\                                //
        \\                              //
         \=> [Canonical Entity: Apple Inc.]

Similarly, to maintain document coherence and narrative flow, Yongjing Yin et al. introduce a sentence-entity graph approach in Graph-based Neural Sentence Ordering. Instead of allowing a fully-connected attention mechanism to create a noisy web of associations, they restrict connections to explicit coreference chains. This structural constraint prevents the model from losing track of the subject when a narrative switches to synonyms or pronominal references.

By treating coreference not as a separate preprocessing step, but as a fundamental geometric constraint on how information flows through a network, these approaches are beginning to close the gap—moving NLP closer to the human capacity for deep, document-level comprehension.

History & citationVersion 2 · 2026-10-04

Current · Permanent link

  1. Version 2 (latest) · 2026-10-04 · 393 papers
  2. Version 1 · 2026-10-04 · 419 papers
Cite this direction

Latest version (a living page):

@misc{lacuna_coreference_resolution_gap_00578143,
  title = {coreference resolution gap},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/coreference-resolution-gap/rdr_00578143541d47a2b2b707f8ee415db9},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 2, frozen text):

@misc{lacuna_coreference_resolution_gap_00578143_v2,
  title = {coreference resolution gap},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/coreference-resolution-gap/rdr_00578143541d47a2b2b707f8ee415db9/v/2},
  note = {Version 2, frozen 2026-10-04. Accessed 2026-11-20},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake