AKEM: Aligning Knowledge Base to Queries with Ensemble Model for Entity Recognition and Linking

Di Lu, Zhongping Liang, Caixia Yuan, Xiaojie WangarXiv 2023 · DOI · arXiv

Abstract

AKEM (Aligning Knowledge Base to Queries with Ensemble Model) is a two-stage framework designed to tackle the "needle in a haystack" problem of identifying and linking entities within short, ambiguous Chinese search queries. By combining aggressive knowledge base expansion with an ensemble of regression models (SVR and MART), the system significantly boosts recall and achieves a competitive F1 score of 0.535, ranking 4th in the NLPCC 2015 evaluation.

Introduction

Entity Recognition and Linking (ERL) is a critical task in information retrieval, particularly for short search queries. Unlike long-form articles, search queries provide minimal context, making disambiguation extremely difficult. For example, the term "苹果" (Apple) could refer to the fruit or the technology company. In the Chinese language context, this is further complicated by word segmentation issues, entity name multiplicity (e.g., abbreviations like "中纪委"), and the "NIL" problem where entities are missing from the reference knowledge base (KB).

Methodology

The AKEM framework operates in two distinct phases: improving recall through knowledge base expansion and filtering results via an ensemble scoring model.

The AKEM framework architecture, illustrating the two-phase process of improving recall (Part I) and filtering results (Part II).

Phase I: Improving Recall

Phase I focuses on ensuring no potential entity is missed by "fattening" the reference knowledge base. The authors normalized English entity names, stripped brackets from titles, and used heuristic inference to identify locations (e.g., inferring a city based on the suffix "市"). They also employed regular expressions to mine nicknames and aliases from entity descriptions. For candidate generation, the system performs word segmentation and sends terms to the Baidu search engine, matching the top 10 results against the expanded KB to generate a broad candidate list.

Phase II: Filtering and Ranking

Once candidates are generated, the system must prune the noise. Nine specific features are extracted for each mention-candidate pair, including character-level similarity and semantic similarity using Word2Vec embeddings trained on the NLPCC 2015 data. The core of the ranking system is an ensemble of Support Vector Regression (SVR) and Multiple Additive Regression Trees (MART). The scores from both models are averaged to create a final ranking score, followed by rule-based refinement to filter out single Chinese characters or resolve duplicate links.

Experimental Results

The AKEM system was evaluated using the datasets provided by the NLPCC 2015 Entity Recognition and Linking Challenge, which included a large-scale Chinese KB, a training set of 159 labeled queries, and an evaluation set of 3,849 queries.

Comparison of AKEM performance metrics against the average performance of participants in the NLPCC 2015 challenge.

As shown in the results table, AKEM demonstrated strong performance, particularly in recall. The system achieved an F1 score of 0.535 and a recall of 0.656, both significantly higher than the participant averages of 0.423 and 0.497, respectively. The system secured 4th place overall among all participating teams.

Conclusion

AKEM demonstrates that for short-query entity linking, prioritizing recall through KB expansion and external search results is a pragmatic and effective strategy. The ensemble of SVR and MART provides a robust mechanism for filtering the resulting noise. Future work will focus on deeper semantic understanding and more sophisticated methods for clustering "NIL" entities that do not exist in the reference knowledge base.

Summary updated

Abstract

This paper presents a novel approach to address the Entity Recognition and Linking Challenge at NLPCC 2015. The task involves extracting named entity mentions from short search queries and linking them to entities within a reference Chinese knowledge base. To tackle this problem, we first expand the existing knowledge base and utilize external knowledge to identify candidate entities, thereby improving the recall rate. Next, we extract features from the candidate entities and utilize Support Vector Regression and Multiple Additive Regression Tree as scoring functions to filter the results. Additionally, we apply rules to further refine the results and enhance precision. Our method is computationally efficient and achieves an F1 score of 0.535.
BibTeX
@misc{lu2023akem,
  author = {Di Lu and Zhongping Liang and Caixia Yuan and Xiaojie Wang},
  title = {{AKEM: Aligning Knowledge Base to Queries with Ensemble Model for Entity Recognition and Linking}},
  year = {2023},
  eprint = {2309.06175},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2309.06175},
}
Figure