Low-Rank Subspaces for Unsupervised Entity Linking

Akhil Arora, Alberto García-Durán, Robert West

Abstract

Entity Linking (EL) is the task of mapping ambiguous textual mentions to unique entries in a Knowledge Base. While supervised methods dominate the field, they are constrained by the need for extensive annotated data. This paper presents EIGENTHEMES, a lightweight, fully unsupervised approach that leverages the geometric properties of entity embeddings. By applying Singular Value Decomposition (SVD) to document-level candidate sets, EIGENTHEMES identifies a low-rank subspace where correct entities naturally cluster. Our results show that this method outperforms previous unsupervised state-of-the-art systems by over 15% in precision while being up to 100x faster.

Introduction

The challenge of Entity Linking lies in ambiguity; a mention like "Michael Jordan" could refer to a legendary basketball player, a prominent computer scientist, or a racing driver. Supervised systems mitigate this by learning from large-scale human annotations, but this approach fails in niche or private domains where such data is unavailable. Unsupervised EL remains a difficult frontier as it must rely solely on the Knowledge Base (KB) structure and entity names without prior knowledge of mention frequency or context-specific patterns.

Methodology

EIGENTHEMES is built on the principle of topical coherence: entities mentioned within the same document are likely to be semantically related. In the vector space of entity embeddings, these related "gold" entities tend to reside within a shared low-rank subspace.

Figure 1: Illustration of the EIGENTHEMES approach. Gold entities (orange triangles) for mentions like 'Michael Jordan' and 'Science' cluster within a low-rank subspace, while incorrect candidates (blue circles) are scattered.

The EIGENTHEMES pipeline consists of four main stages:

  1. Candidate Generation: Identifying all potential KB matches for each mention via string matching.
  2. Embedding Retrieval: Fetching pre-computed graph embeddings (e.g., DeepWalk) for all candidates.
  3. Subspace Learning: Constructing a matrix of these embeddings and using SVD to find the "eigenthemes"—the principal components that define the document's latent topics.
  4. Scoring: Ranking candidates based on their proximity to the learned subspace. A weighted variant further incorporates KB-derived signals like entity popularity to guide the decomposition.

Evaluation and Results

We evaluated EIGENTHEMES across several standard benchmarks, including AIDA-CoNLL and the WNED datasets (Wiki and Clueweb). These datasets provide a rigorous test of the system's ability to handle "Hard" mentions, where the correct entity is not the most popular candidate.

Figure 2: Precision@1 comparison across Wiki, Clueweb, and Wikilinks datasets. EIGENTHEMES (Eigen, red bar) consistently outperforms other unsupervised methods like τMIL-ND.

EIGENTHEMES demonstrates significant gains in both accuracy and efficiency:

  • Accuracy: On the CoNLL-Test set, the method achieved a Precision@1 of 0.617, compared to 0.451 for the previous unsupervised leader (τMIL-ND).
  • Efficiency: The reliance on linear algebra rather than deep learning allows EIGENTHEMES to process documents 10x to 100x faster than neural baselines.
  • Robustness: With only one primary hyperparameter ($k$), the model is highly portable across different domains.

Conclusion and Future Work

EIGENTHEMES demonstrates that topical relatedness is a structural property that can be effectively captured through low-rank subspaces. While the system is currently limited by the coverage of the underlying Knowledge Graph and the recall of the candidate generator, its success suggests that topical relatedness is a powerful structural signal for disambiguation. Future work will explore multi-lingual applications where the language-agnostic nature of graph embeddings can be fully leveraged.

Summary updated

Abstract

Entity linking is an important problem with many applications. Most previous solutions were designed for settings where annotated training data is available, which is, however, not the case in numerous domains. We propose a light-weight and scalable entity linking method, Eigenthemes, that relies solely on the availability of entity names and a referent knowledge base. Eigenthemes exploits the fact that the entities that are truly mentioned in a document (the "gold entities") tend to form a semantically dense subset of the set of all candidate entities in the document. Geometrically speaking, when representing entities as vectors via some given embedding, the gold entities tend to lie in a low-rank subspace of the full embedding space. Eigenthemes identifies this subspace using the singular value decomposition and scores candidate entities according to their proximity to the subspace. On the empirical front, we introduce multiple strong baselines that compare favorably to (and sometimes even outperform) the existing state of the art. Extensive experiments on benchmark datasets from a variety of real-world domains showcase the effectiveness of our approach.
BibTeX
@inproceedings{arora2021low,
  author = {Akhil Arora and Alberto García-Durán and Robert West},
  title = {{Low-Rank Subspaces for Unsupervised Entity Linking}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2021},
  doi = {10.18653/v1/2021.emnlp-main.634},
}
Figure