Introduction
Entity Linking (EL) is the process of recognizing mentions in text and mapping them to unique entries in a knowledge base. Traditional approaches often rely on predefined candidate sets and a two-step detection-then-disambiguation pipeline. While recent autoregressive models treat EL as a translation task, they often lack flexibility and struggle with one-shot prediction errors.
This paper introduces a multi-task autoregressive model that eliminates candidate lists and incorporates a novel re-ranking mechanism. By training with auxiliary tasks—Mention Detection and Match Prediction—the authors achieve state-of-the-art results on news and biomedical datasets.
Methodology
The system is built on the BART Large encoder-decoder transformer. The innovation lies in a Multi-Task Learning architecture that shares an encoder across three distinct tasks:
- Autoregressive Entity Linking (Main Task): The model generates the input sentence, replacing mentions with canonical entity names from a knowledge base.
- Mention Detection (MD): A token-wise classification task that teaches the model to identify entity spans, focusing its attention on relevant text.
- Match Prediction (MP): A novel task where the model evaluates sentence pairs to decide if the entities in the output correctly match the mentions in the source.
At inference time, the model uses beam search to generate several candidate sentences. The Match Prediction module then re-ranks these candidates, selecting the one with the highest match score. This "second-guessing" allows the model to bridge the gap between its initial language model probability and the actual correct link.
Key Results and Insights
The model established new state-of-the-art (SOTA) benchmarks:
- AIDA-CoNLL (News): Achieved a Micro-F1 score of 85.7, outperforming both previous autoregressive models and those using predefined candidate sets.
- COMETA (Biomedical): Achieved an Acc@1 of 32.4, a significant jump over the previous best of 30.9.
A key insight from the study is that the performance boost is structural rather than purely data-driven. Experiments showed that even with a 1:1 ratio of auxiliary data to main task data, the model performed better, suggesting that the auxiliary tasks help learn more robust internal representations. Furthermore, the model learns the entity mapping internally, treating the knowledge base as a language it is learning to speak, which removes the need for a massive searchable index at runtime.
Limitations and Ethical Considerations
Despite its success, the model faces challenges with sequence length in long news articles, often requiring text to be chunked. Additionally, the multi-task setup and re-ranking phase add computational overhead. In high-stakes domains like medicine, the authors caution that the model should not be used without human supervision, as mislinked entities could lead to medical misunderstandings. Future work will explore scaling to larger knowledge bases and optimizing the re-ranking logic for faster inference.
Summary updated