End-to-End Entity Linking with Hierarchical Reinforcement Learning

Lihan Chen, Tinghui Zhu, Jingping Liu, Jiaqing Liang, Yanghua Xiao

Introduction

Entity Linking (EL) is a fundamental task in natural language processing that involves identifying text segments (mentions) and mapping them to corresponding entries in a Knowledge Graph (entities). Traditionally, EL is decomposed into two distinct phases: Mention Detection (MD) and Entity Disambiguation (ED).

However, conventional approaches suffer from two primary limitations. First, inter-dependency issues: pipeline methods treat MD and ED as isolated steps, failing to leverage the fact that a successful entity link can provide critical feedback for refining mention boundaries. Second, intra-dependency issues: most models treat mentions within a document as independent tasks, ignoring the contextual relationships between entities (e.g., linking "Jobs" to Steve Jobs significantly increases the likelihood that "Apple" refers to the technology company).

Hierarchical Reinforcement Learning Framework

To address these challenges, this paper proposes a novel Hierarchical Reinforcement Learning (HRL) framework. This approach models EL as a sequential decision-making process, mimicking the cognitive workflow of a human annotator who identifies and links mentions iteratively.

Overview of the Hierarchical Reinforcement Learning framework for Entity Linking.

The architecture consists of two levels:

  1. High-level Policy (The Strategist): Responsible for sequential MD. It selects the next text span to be linked using a Longformer encoder to maintain global document context.
  2. Low-level Policy (The Executor): Performs ED for the selected mention. It uses a generative approach to autoregressively produce the entity name token-by-token via an LSTM.

The high-level policy is optimized to maximize the total EL reward. If the low-level policy fails to link a mention correctly, the high-level policy receives a penalty, effectively teaching it to identify more "linkable" and accurate mention boundaries. To stabilize training, the authors introduced a discriminator-based value function that estimates the potential success of remaining text spans.

Key Results and Evaluation

The model was evaluated on the AIDA-CoNLL benchmark and several out-of-domain datasets. The HRL approach demonstrated significant improvements over existing pipeline and end-to-end models.

Comparison of the proposed HRL model against pipeline and end-to-end baselines on the AIDA-CoNLL dataset.

As shown in the results table, the model achieved a new state-of-the-art 87.6 Micro-F1 score on the AIDA-CoNLL test set. Beyond in-domain performance, the HRL framework exhibited superior generalization, achieving a +1.6 improvement in macro-averaged Micro-F1 across four out-of-domain datasets. Ablation studies further confirmed that "Hierarchical Inference" (HI)—making decisions step-by-step—outperforms "Pipeline Inference" (PI), validating the importance of capturing intra-dependencies.

Conclusion and Future Work

By framing Entity Linking as a hierarchical, sequential task, this work successfully captures the complex dependencies between mentions and entities. While the model achieves state-of-the-art accuracy, its sequential nature results in slower inference speeds compared to parallel models. Future research may focus on optimizing this latency or replacing the low-level LSTM with larger pre-trained transformers like BART to further enhance disambiguation capabilities.

Summary updated

Abstract

Entity linking (EL) is the task of linking the text segments to the referring entities in the knowledge graph, typically decomposed into mention detection, and entity disambiguation. Compared to traditional methods treating the two tasks separately, recent end-to-end entity linking methods exploit the mutual dependency between mentions and entities to achieve better performance. However, existing end-to-end EL methods have problems utilizing the dependency of mentions and entities in the task. To this end, we propose to model the EL task as a hierarchical decision-making process and design a hierarchical reinforcement learning algorithm to solve the problem. We conduct extensive experiments to show that the proposed method achieves state-of-the-art performance in several EL benchmark datasets. Our code is publicly available at https://github.com/lhlclhl/he2eel.
BibTeX
@inproceedings{chen2023end,
  author = {Lihan Chen and Tinghui Zhu and Jingping Liu and Jiaqing Liang and Yanghua Xiao},
  title = {{End-to-End Entity Linking with Hierarchical Reinforcement Learning}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2023},
  doi = {10.1609/aaai.v37i4.25534},
}
Figure