Mitigating Popularity Bias in Entity Disambiguation and Retrieval

This direction is split into other directions: popularity bias (popularity-based entity disambiguation), popularity bias (entity popularity bias). Its last version stays readable and citable here.

Version 1 of 194 papers131 conceptsfrozen 2026-10-04

In natural language processing, not all entities are created equal. While a model might easily identify "Paris" as the capital of France, it often struggles when a text refers to Paris, Texas, or a person named Paris. This systemic failure is known as popularity bias: the tendency for models to favor famous, "head" entities over the vast "long tail" of obscure ones.

Across decades of research, this bias has been identified as a primary bottleneck in tasks ranging from entity linking to open-domain question answering.

The Mechanism of Entity Overshadowing

The core of the problem lies in entity overshadowing. When a name is ambiguous, models often rely on a "global popularity prior"—a statistical shortcut that assumes the most famous entity is the intended one. Vera Provatorova and colleagues introduced the ShadowLink benchmark to demonstrate this effect, showing that state-of-the-art models fail to identify "shadow" entities even when the local context explicitly supports them Robustness Evaluation of Entity Disambiguation Using Prior Probes: the Case of Entity Overshadowing.

This is not just a minor error; it is a fundamental reliance on statistical guessing over linguistic understanding. As Johannes Hoffart noted in the development of the AIDA framework, systems that default to the most famous candidate fail to capture the specific, niche contexts where a less common entity is the correct referent Robust Disambiguation of Named Entities in Text.

Why Models Take Shortcuts

Why do models "cheat" by using popularity? The answer lies in how they are trained. Panuthep Tasawong and Peerat Limkonchotiwat argue that models engage in shortcut learning, prioritizing the most frequent entity associated with a name because it yields high accuracy on standard, biased benchmarks Efficient Overshadowed Entity Disambiguation by Mitigating Shortcut Learning.

Furthermore, the data used to train these models is often self-reinforcing. Quzhe Huang found that the "recommend-revise" annotation process used in datasets like DocRED misses two-thirds of actual relation instances, systematically favoring popular entities already present in knowledge bases Does Recommend-Revise Produce Reliable Annotations? An Analysis on Missing Instances in DocRED. This creates a feedback loop where models are trained on data that ignores the long tail, leading to poor generalization in the real world.

The Failure of Dense Retrieval

The popularity problem extends beyond disambiguation into information retrieval. While neural dense retrievers (like DPR) are powerful, they often underperform traditional keyword-matching methods (like BM25) when queries involve rare entities.

Christopher Sciavolino and Danqi Chen demonstrated this with the EntityQuestions benchmark, revealing that dense retrievers rely more on memorizing frequent entity-pattern associations than on learning robust retrieval mechanisms Simple Entity-Centric Questions Challenge Dense Retrievers. When a query asks about a "tail" entity, the model’s parametric bias pulls it toward a more famous "name twin," resulting in incorrect document retrieval Evaluating Entity Disambiguation and the Role of Popularity in Retrieval-Based NLP.

Strategies for Robustness

To move beyond popularity-based heuristics, researchers have proposed several mechanisms to force models to prioritize context and structure:

  1. Structured Reasoning: By explicitly reasoning over the relations in a Knowledge Base (KB), models can find evidence for rare entities that share names with famous ones. Tom Ayoola showed that using relation extraction to identify connections between mentions in a document can lead to massive improvements on rare-entity datasets Improving Entity Disambiguation by Reasoning over a Knowledge Base.
  2. Counterfactual Training: To break the reliance on name-popularity shortcuts, Panuthep Tasawong proposed masking the entity name during training. This forces the model to identify the entity based solely on the surrounding context Efficient Overshadowed Entity Disambiguation by Mitigating Shortcut Learning.
  3. Social and Topical Context: In sparse environments like Twitter, linguistic context may be insufficient. Yi Yang 0038 demonstrated that incorporating the social interests of a user can help disambiguate niche entities that would otherwise be overshadowed by globally popular ones Toward Socially-Infused Information Extraction: Embedding Authors, Mentions, and Entities.
  4. Joint Modeling: Instead of treating entity linking and type inference as separate steps, Hrushikesh Mohapatra proposed joint bootstrapping, where the synergy between identifying an entity's type and its identity helps resolve obscure mentions in the long tail Joint Bootstrapping of Corpus Annotations and Entity Types.

By shifting the focus from global statistics to local evidence and structured knowledge, these approaches aim to build systems that truly understand the text, rather than just following the most popular path.

Papers in this version

Showing 60 of 94 papers.

History & citationVersion 1 · 2026-10-04

Split into other directions · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_mitigating_popularity_bias_in_entity_dis_cb955c95,
  title = {Mitigating Popularity Bias in Entity Disambiguation and Retrieval},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/mitigating-popularity-bias-in-entity-disambiguation-and-retrieval/rdr_cb955c95f4564dc58b227e4596974bba},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_mitigating_popularity_bias_in_entity_dis_cb955c95_v1,
  title = {Mitigating Popularity Bias in Entity Disambiguation and Retrieval},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/mitigating-popularity-bias-in-entity-disambiguation-and-retrieval/rdr_cb955c95f4564dc58b227e4596974bba/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}