Efficient One-Pass End-to-End Entity Linking for Questions

Belinda Z. Li, Sewon Min, Srinivasan Iyer, Yashar Mehdad, Wen-tau Yih

Abstract

We present ELQ, a fast end-to-end entity linking model for questions, which uses a bi-encoder to jointly perform mention detection and linking in one pass. Evaluated on WebQSP and GraphQuestions with extended annotations that cover multiple entities per question, ELQ outperforms the previous state of the art by a large margin of +12.7% and +19.6% F1, respectively. With a very fast inference time (1.57 examples/s on a single CPU), ELQ can be useful for downstream question answering systems. In a proof-of-concept experiment, we demonstrate that using ELQ significantly improves the downstream QA performance of GraphRetriever.

Introduction

Entity Linking (EL) is the process of recognizing a "mention" in text (e.g., "shaq") and mapping it to a specific entry in a knowledge base (e.g., Shaquille O'Neal). While EL is a relatively mature field for long, well-formatted documents like news articles, it remains a significant challenge for short, noisy user queries. Questions often lack capitalization, punctuation, and clear context (e.g., "who is the senator of georgia").

Existing EL systems typically fall into three categories: two-stage pipelines that propagate errors from mention detection to linking, disambiguation-only models that require pre-identified mentions, or slow cross-encoders that are computationally prohibitive for real-time applications. ELQ (Efficient Entity Linking for Questions) addresses these issues by performing mention detection and entity disambiguation in a single, efficient pass.

Methodology: The Bi-Encoder Architecture

ELQ utilizes a bi-encoder architecture based on BERT, which decouples the encoding of the question from the encoding of the entities. This separation allows for massive speedups through pre-computation and vector search.

ELQ architecture using a bi-encoder for joint mention detection and entity linking.

The system consists of two primary components:

  • Question Encoder: A BERT model that produces token-level embeddings. It simultaneously predicts the start and end of mention spans and generates a "mention representation" by averaging the tokens within a span.
  • Entity Encoder: A BERT model that pre-computes embeddings for every entity in Wikipedia based on their title and a short description.

During inference, ELQ identifies potential mention spans in a single forward pass. It then performs an inner product between the mention representations and the pre-computed entity embeddings. Using FAISS for efficient similarity search, the "linking" step becomes nearly instantaneous.

Experimental Results

The authors evaluated ELQ on two primary benchmarks with extended end-to-end annotations: WebQSP_EL (short Google search queries) and GraphQ_EL (used for zero-shot evaluation).

Performance comparison of ELQ against baselines on WebQSP and GraphQuestions datasets.

ELQ achieved an F1 score of 89.0% on WebQSP, representing a +12.7% improvement over the previous state-of-the-art (VCG). On the zero-shot GraphQ dataset, it reached 70.3% F1, a +19.6% improvement. Beyond accuracy, ELQ is highly efficient, processing 1.57 questions per second on a single CPU, which is roughly twice as fast as neural baselines like BLINK.

Key Insights and Downstream Impact

A major finding of this work is the synergy of joint training. By training mention detection and entity linking together, the model learns to identify mentions partly by recognizing which spans correspond to known entities in the database.

Furthermore, the paper demonstrates that improved EL directly translates to better performance in complex downstream tasks. Replacing standard linkers with ELQ in a GraphRetriever system improved Exact Match (EM) scores by up to 6% across three open-domain datasets (WQ, NQ, TQA).

Limitations and Future Work

Despite its efficiency, the BERT forward pass remains the primary bottleneck, accounting for approximately 0.68s of inference time. Future iterations could utilize distilled models to further reduce latency. Qualitative error analysis also revealed that many "errors" were technically correct but differed from human annotations (e.g., finding a more specific entity). Finally, some queries remain inherently ambiguous without temporal or external context, such as identifying a "senator" without a specific date.

While optimized for questions, the one-pass bi-encoder approach holds promise for generalization to longer, well-formed documents in future research.

Summary updated

Abstract

We present ELQ, a fast end-to-end entity linking model for questions, which uses a biencoder to jointly perform mention detection and linking in one pass. Evaluated on WebQSP and GraphQuestions with extended annotations that cover multiple entities per question, ELQ outperforms the previous state of the art by a large margin of +12.7% and +19.6% F1, respectively. With a very fast inference time (1.57 examples/s on a single CPU), ELQ can be useful for downstream question answering systems. In a proof-of-concept experiment, we demonstrate that using ELQ significantly improves the downstream QA performance of GraphRetriever (arXiv:1911.03868). Code and data available at https://github.com/facebookresearch/BLINK/tree/master/elq
BibTeX
@inproceedings{li2020efficient,
  author = {Belinda Z. Li and Sewon Min and Srinivasan Iyer and Yashar Mehdad and Wen-tau Yih},
  title = {{Efficient One-Pass End-to-End Entity Linking for Questions}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2020},
  url = {https://openreview.net/forum?id=L-CtqyNYSAwy},
}
Figure