Abstract
We propose an autoregressive entity linking model trained with two auxiliary tasks that learns to re-rank generated samples at inference time. Our proposed novelties address two weaknesses in the literature. First, while recent methods propose learning mention detection and entity candidate selection, they often rely on predefined sets of candidates. We use an encoder-decoder autoregressive framework to bypass this need, training mention detection as an auxiliary task instead. Second, to address the common issue of prediction errors in top-1 outputs, we introduce a match prediction auxiliary task to facilitate re-ranking. Without the use of a knowledge base or candidate sets, our model sets a new state of the art on two benchmark datasets: COMETA (biomedical) and AIDA-CoNLL (news). Ablation studies demonstrate that both auxiliary tasks increase performance, with re-ranking serving as a critical factor. Furthermore, low-resource experiments suggest that the main task benefits from the knowledge learned by auxiliary tasks beyond mere data augmentation.
Introduction
Entity Linking (EL) is the fundamental task of mapping textual mentions to unique entries in a knowledge base. Traditionally, this involves a two-stage pipeline: Mention Detection (MD) followed by Entity Disambiguation (ED). While effective, this approach often relies on restrictive candidate sets—predefined lists of potential matches—that limit flexibility and introduce noise from external lookup tables.
Recent shifts toward autoregressive EL treat the problem as a sequence-to-sequence generation task, "translating" source text into a version where mentions are replaced by canonical entity names. However, these models frequently suffer from the "Top-1" error: the correct entity is often present within the model's top 10 generated candidates but is not ranked as the primary choice. This paper introduces a multi-task architecture designed to eliminate candidate dependencies and implement a robust re-ranking mechanism.
Methodology: Multi-Task Autoregressive Learning
The proposed model utilizes a shared BART encoder with three distinct decoders, each optimized for a specific facet of the EL problem.
1. The Entity Linking (EL) Decoder
This is the primary task. The decoder generates the target sequence where mentions are substituted with their canonical names. By learning the mapping directly within the model parameters, the system bypasses the need for external candidate sets or knowledge base lookups during inference.
2. The Mention Detection (MD) Decoder
To improve the model's structural understanding of entities, the MD decoder performs token-wise classification (e.g., "E" for entity, "O" for other). This auxiliary task teaches the model to identify the boundaries of mentions without requiring a separate pipeline step, reinforcing the encoder's representation of what constitutes an entity.
3. The Match Prediction (MP) Decoder
The MP decoder acts as a binary classifier. It evaluates the input sentence alongside a generated sample to predict whether the generation is a correct match. This task is the foundation for the model's re-ranking capability.
Inference and Re-ranking Strategy
During inference, the model generates $k$ possible candidates using beam search. Rather than simply selecting the candidate with the highest language modeling probability, the system uses the Match Prediction module to score and re-rank the candidates. This allows the model to "self-correct," identifying the most accurate match even if it was not the most likely sequence according to the base generator.
Experimental Results
The model was evaluated on two diverse benchmarks: AIDA-CoNLL (news domain) and COMETA (biomedical domain). The latter is particularly challenging due to its "zero-shot" split, where test entities are absent from the training set.
| Dataset | Metric | Previous Best (Autoregressive) | Our Model |
|---|---|---|---|
| AIDA-CoNLL | Micro-F1 | 83.7 | 85.7 |
| COMETA | Acc@1 | 30.9 | 32.4 |
Key Findings
- State-of-the-Art Performance: The model established new benchmarks on both datasets, outperforming traditional embedding-based methods and previous autoregressive models.
- The Impact of Re-ranking: Ablation studies confirmed that removing the re-ranking step significantly degraded performance. The ability to evaluate candidates holistically via the MP decoder is essential for bridging the gap between "Top-10" and "Top-1" accuracy.
- Low-Resource Synergy: Experiments with limited training data showed that auxiliary tasks provide a "teaching" effect. The knowledge gained from learning to detect mentions (MD) directly improved the model's ability to disambiguate them (EL), even when data was sparse.
Discussion and Limitations
While the multi-task approach offers significant accuracy gains, it introduces computational overhead. Running three decoders and re-ranking $k$ samples increases latency compared to single-pass models. Additionally, while the model excels in specific domains, further research is required to determine how well this generative approach scales to massive, multi-million entity knowledge bases like Wikidata.
Finally, the authors emphasize ethical considerations in the biomedical domain. While the model shows high accuracy on COMETA, it is not a substitute for professional medical judgment and should be used as a supportive tool rather than an automated diagnostic system.
Conclusion
By framing Entity Linking as a multi-task autoregressive problem, this work demonstrates that mention detection and match prediction are not just related tasks, but essential components for high-accuracy disambiguation. The transition from simple generation to a "generate-and-check" framework via re-ranking provides a robust path forward for candidate-free entity linking.
Summary updated