Resolving Unseen Entities with Textual Descriptions

51 concepts42 papers

The Limits of Memorized Entity Linking

Traditional entity linking (EL) systems treat the problem of mapping text mentions to a knowledge base (KB) as a massive multi-class classification task. If a model encounters the mention "Michael Jordan" in a sports article, it relies on pre-computed frequency statistics and memorized entity IDs to map the mention directly to the basketball player's unique database entry.

However, this paradigm breaks down completely when encountering entities or shifting to new domains, such as specialized biomedical databases or private corporate repositories. As noted in Memorisation versus Generalisation in Pre-trained Language Models by Michael Tänzer et al., pre-trained language models exhibit a "pattern-first" bias that excels at memorization but struggles to generalize when there is no overlap between the entities seen during training and those encountered during testing. When frequency statistics and entity IDs are unavailable, traditional classifiers fail.

To solve this, a new paradigm has emerged: Zero-Shot Entity Linking (ZEL). Instead of learning discrete entity IDs, models learn to match the context of a mention with the textual description of an entity in a shared vector space.

The Two-Stage Dense Retrieval Architecture

The foundational mechanism for zero-shot entity linking relies on representing both mentions and entities purely through text. In Scalable Zero-shot Entity Linking with Dense Entity Retrieval, Ledell Wu et al. introduced a scalable two-stage framework that decouples candidate retrieval from deep contextual re-ranking.

1. The Bi-Encoder for Fast Retrieval

In the first stage, a bi-encoder independently processes the mention context and the entity description using two separate transformer backbones.

  • The mention encoder takes the source sentence (with special tokens highlighting the mention span) and outputs a dense vector $\mathbf{y}_m$.
  • The entity encoder takes the entity's title and textual description from the knowledge base and outputs a dense vector $\mathbf{y}_e$.

Because these representations are independent, entity vectors can be pre-computed and indexed using approximate nearest neighbor search algorithms. At inference time, the model computes the dot product $\mathbf{y}_m \cdot \mathbf{y}_e$ to instantly retrieve the top-$k$ candidate entities out of millions.

2. The Cross-Encoder for Deep Interaction

While the bi-encoder is fast, it cannot model token-to-token interactions between the mention context and the entity description. To resolve subtle ambiguities, the second stage employs a cross-encoder.

The cross-encoder concatenates the mention context and a candidate entity description into a single sequence, feeding it into a shared transformer. This allows full self-attention across both texts. The model then outputs a scalar score indicating the quality of the match. By running this expensive operation only on the top-$k$ candidates retrieved by the bi-encoder, the system achieves high precision without sacrificing scalability.

To further improve this architecture under severe domain shifts, the authors of A Transformational Biencoder with In-Domain Negative Sampling for Zero-Shot Entity Linking (Kai Sun et al.) proposed inserting a learnable transformation layer to explicitly model domain shifts, combined with an in-domain negative sampling strategy to sharpen the bi-encoder's discrimination boundaries.

Regularizing with Fine-Grained Semantic Types

If a model is trained purely to match text descriptions, it can still fall into the trap of memorizing specific mention-entity pairs from the training set. To force the model to learn transferable semantic principles, researchers have introduced auxiliary typing constraints.

In Fine-Grained Entity Typing for Domain Independent Entity Linking, Yasumasa Onoe et al. reframed entity linking as a massive-scale fine-grained typing problem. Instead of predicting an entity ID, their model is trained to predict over 60,000 Wikipedia-derived types. For example, instead of mapping "Python" to a specific ID, the model learns to recognize the fine-grained types /software/programming_language or /animal/reptile/snake based on context. This acts as a powerful regularizer: by learning to predict abstract type profiles, the model generalizes seamlessly to entirely unseen entities that share those underlying semantic properties.

Similarly, in the biomedical domain, Dongfang Xu et al. presented A Generate-and-Rank Framework with Semantic Type Regularization for Biomedical Concept Normalization, which integrated a Semantic Type Regularizer into a generate-and-rank framework. This ensures that even if a specific medical concept was never seen during training, the model constrains its predictions to the correct ontological category, preventing catastrophic mismatches.

Collective Disambiguation via Mention-Mention Coreference

Resolving a single ambiguous mention (like "AD" in a medical text, which could mean Alzheimer’s Disease or Atopic Dermatitis) in isolation is incredibly difficult in a zero-shot setting. However, documents rarely contain just one mention; instead, they feature multiple related mentions of the same entity.

To exploit this, Rico Angell et al. proposed a constrained clustering approach in Clustering-based Inference for Biomedical Entity Linking. Instead of linking mentions independently, their framework computes affinities between different mentions (mention-mention) as well as between mentions and entities (mention-entity). Mentions that refer to the same underlying entity are clustered together into a single representation.

Building on this intuition, Dhruv Agarwal et al. explicitly modeled coreference in Entity Linking via Explicit Mention-Mention Coreference Modeling. By training dual-encoders to construct directed minimum spanning trees across mentions and entities, a highly ambiguous mention (e.g., "the drug") can inherit the clear contextual information of a coreferent mention elsewhere in the text (e.g., "Acetaminophen"), dramatically improving zero-shot candidate retrieval.

Hybrid Architectures: Bridging Neural Semantics and Symbolic Fallbacks

While dense retrieval handles semantic variations well, pure neural models sometimes struggle with exact string matches or rare technical jargon that was absent from the pre-training corpus.

To address this, Hayate Iso et al. developed a hybrid approach in End-to-end Biomedical Entity Linking with Span-based Dictionary Matching. They integrated span-based neural representations from PubMedBERT with traditional TF-IDF-based dictionary scores. By combining character-level dictionary matching with contextual neural embeddings, the model creates a robust fallback mechanism: if the neural encoder is uncertain about an unseen entity, the symbolic dictionary component can still resolve the mention via exact or partial string matching against the reference database.

Furthermore, to handle completely novel knowledge base structures, Miguel Ballesteros et al. designed a framework in Linking Entities to Unseen Knowledge Bases with Arbitrary Schemas. By treating database attributes as special tokens and utilizing attribute shuffling during training, their model learns to link mentions to unseen knowledge bases with entirely arbitrary schemas, proving that zero-shot generalization can extend beyond unseen entities to entirely unseen database structures.

Key Foundational Work

Multi-level cross-lingual attentive neural architecture for low resource name tagging (2017) — contributed 1 concept elements to this direction.
Temporal dynamics of semantic relations in word embeddings: an application to predicting armed conflict participants (2017) — contributed 1 concept elements to this direction.
Copy or Coincidence? A Model for Detecting Social Influence and Duplication Events (2013) — contributed 1 concept elements to this direction.
History & citationVersion 2 · 2026-10-04

Current · Permanent link

  1. Version 2 (latest) · 2026-10-04 · 42 papers
  2. Version 1 · 2026-10-04 · 69 papers
Cite this direction

Latest version (a living page):

@misc{lacuna_zero_shot_entity_linking_76a30efe,
  title = {zero-shot entity linking},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/zero-shot-entity-linking/rdr_76a30efe862c43cbad70139091eb447a},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 2, frozen text):

@misc{lacuna_zero_shot_entity_linking_76a30efe_v2,
  title = {zero-shot entity linking},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/zero-shot-entity-linking/rdr_76a30efe862c43cbad70139091eb447a/v/2},
  note = {Version 2, frozen 2026-10-04. Accessed 2026-11-19},
}

See something wrong? Report a mistake