Abstract
Entity Linking (EL) is a critical task in natural language processing that involves grounding textual mentions to specific entities in a knowledge graph. While supervised models achieve high accuracy, they require extensive labeled data, which is often unavailable for niche domains. We present EIGENTHEMES, a novel unsupervised EL method that leverages the geometric properties of entity embeddings. By identifying low-rank subspaces that capture the topical essence of a document, EIGENTHEMES achieves state-of-the-art performance while being significantly faster than existing unsupervised approaches.
Introduction
The challenge of Entity Linking lies in disambiguation. For instance, a mention of "Michael Jordan" could refer to the legendary basketball player, the prominent computer scientist, or a racing driver. Supervised models excel at this by learning from thousands of examples, but their "data-hungry" nature limits their applicability in specialized fields like medicine or law. Unsupervised Entity Linking aims to solve this by using only a reference Knowledge Graph (KG) without any labeled training data.
Methodology: The EIGENTHEMES Approach
EIGENTHEMES treats disambiguation as a geometric problem. The core intuition is that the correct entities within a single document are topically related and thus should cluster together in a high-dimensional embedding space.
The process follows four main steps:
- Candidate Generation: Potential entities are identified for each mention using surface name matching and ranked by their popularity (degree) in the KG.
- Subspace Learning: Using Singular Value Decomposition (SVD), the model identifies a low-rank subspace—the "eigenthemes"—from the embeddings of all candidate entities in the document.
- Scoring and Weighting: Candidates are scored based on their proximity to this learned subspace. To mitigate noise from irrelevant candidates, the model incorporates "priors" like entity popularity as weights during the SVD process.
- Disambiguation: The entity with the highest alignment to the document's eigenthemes is selected as the link.
Experimental Setup
The authors evaluated EIGENTHEMES on five diverse benchmark datasets:
- CoNLL-Val & CoNLL-Test: Standard benchmarks for general EL.
- WNED-Wiki & WNED-Clueweb: Datasets designed to test performance on less popular entities.
- Wikilinks-Random: A challenging dataset consisting of Wikipedia tables, which lack traditional sentence context.
Key Insights: The Gold Cluster Hypothesis
A major contribution of this work is the data-driven validation of the "Gold Cluster" Hypothesis. The researchers demonstrated that correct entities in a document are significantly more related to each other than random subsets of candidates. By using SVD to find this global topical signature, EIGENTHEMES can reason about context more effectively than models relying on local word windows.
Furthermore, the model shows remarkable robustness to "hard" mentions—cases where the correct entity is not the most popular one. While many baselines default to the most frequent entity, EIGENTHEMES uses the document's geometric context to identify the correct, albeit less popular, link.
Results
EIGENTHEMES significantly outperforms previous unsupervised state-of-the-art models in both accuracy and efficiency.
- Accuracy: On the CoNLL-Test set, EIGENTHEMES achieved a Precision@1 of 61.7%, a substantial improvement over the previous best unsupervised model (τMIL-ND) at 45.1%.
- Efficiency: The method is 10 to 100 times faster than competing deep learning approaches. While τMIL-ND required over 200 minutes to process the datasets, EIGENTHEMES completed the task in 2–15 minutes.
- Supervised Integration: When used as a feature in supervised models, EIGENTHEMES provided a 1–2% performance boost, particularly in resolving difficult, context-dependent mentions.
Conclusion and Future Work
EIGENTHEMES demonstrates that low-rank matrix decomposition is a powerful tool for unsupervised entity linking. Its speed and accuracy make it a viable solution for large-scale applications where labeled data is scarce. Future research will explore "inductive learning" to handle entities missing from the initial KG and validate the approach across multiple languages.
Summary updated