Introduction
Training robust Entity Linking (EL) models typically requires high-precision "gold standard" annotations that define the exact start and end of entity mentions. However, achieving this level of precision is labor-intensive and often subjective. This paper investigates the feasibility of "boundary-relaxed" annotation—intentionally allowing for less precise mention boundaries to reduce the human effort required for dataset creation.
Entity Linking is a dual-stage process: first, Named Entity Recognition (NER) identifies a mention in text; second, Entity Disambiguation (ED) maps that mention to a unique entry in a knowledge base. The central question is whether downstream performance remains acceptable if the training data includes "noisy" boundaries—for instance, including adjacent words like "the" or "pain" along with the core entity.
Methodology
The researchers performed a controlled stress test using the AIDA CoNLL-YAGO dataset. They introduced artificial noise by expanding entity boundaries: for a specified percentage of entities (the "noise level"), a random token was added to the left, right, or both sides of the mention.
Three model architectures were evaluated:
- HPAELDC: A state-of-the-art (SoTA) autoregressive model.
- LUKE: A transformer-based model utilizing entity-aware self-attention.
- VanillaNER: A BERT-based baseline representing a lower-complexity approach.
Performance was measured using both strict metrics (requiring exact boundary matches) and relaxed metrics (allowing any overlap) across noise levels from 0% to 100%.
Results and Analysis
The study revealed a critical "breaking point" at approximately 40% noise. Below this threshold, performance across all models remained relatively stable. Beyond it, performance metrics plummeted, particularly for more complex architectures.
Model Robustness vs. Complexity
A surprising finding was that the most sophisticated models, such as LUKE, were the least resilient to noisy boundaries. As noise increased, these models often failed to detect entities entirely. In contrast, the simpler VanillaNER baseline proved more robust; in a relaxed evaluation, it lost only about 15% F-score even when every training instance was noisy. This suggests that simpler models may be better suited for "quick and dirty" datasets.
The Fragility of Disambiguation
The Disambiguation (ED) phase emerged as the primary bottleneck. Most ED algorithms rely on fixed vocabularies of surface forms. When the NER step provides a boundary-expanded mention (e.g., "joint inflammation pain" instead of "joint inflammation"), the ED model often treats it as an out-of-vocabulary (OOV) term. For the LUKE model, disambiguation accuracy for these slightly expanded mentions was a mere 0.015, compared to 0.925 for exact matches.
Discussion and Conclusion
While NER models can be trained to find entities with fuzzy boundaries, the overall EL pipeline remains fragile due to the rigid string-matching requirements of current disambiguation methods. To make boundary-relaxed annotation a viable strategy for reducing human labor, future research must focus on developing ED components capable of approximate string retrieval. If linkers can effectively "ignore" boundary noise, the barrier to creating high-quality EL datasets could be significantly lowered.
Summary updated