Community Question Answering Entity Linking via Leveraging Auxiliary Data

Yuhan Li, Wei Shen, Jianbo Gao, Yadong Wang

Abstract

Entity Linking (EL) is a fundamental NLP task that maps textual mentions to specific entries in a knowledge base. While well-studied in formal texts, EL remains challenging in Community Question Answering (CQA) environments due to short contexts and informal language. This paper proposes a novel framework that leverages auxiliary data—such as topic tags, user history, and parallel answers—to disambiguate mentions. Our approach achieves a state-of-the-art accuracy of 92.02% on the newly introduced QuoraEL dataset.

Introduction

Standard Entity Linking models often struggle with the informal and concise nature of CQA platforms like Quora or Stack Overflow. Mentions like "Roosevelt" or "AGI" are difficult to resolve when the immediate text provides minimal context. However, CQA platforms provide unique metadata that can serve as "side information." For example, a post tagged with "World War II" strongly suggests "Franklin D. Roosevelt" over "Theodore Roosevelt."

Overview of the CQAEL task, illustrating how topic tags, user history, and parallel answers provide critical context for disambiguation.

The authors formalize this as the CQAEL task, focusing on harnessing these auxiliary signals to improve linking accuracy in noisy, low-context environments.

Proposed Methodology: A Dual-Module Framework

The researchers developed a framework consisting of two primary components designed to handle both local context and external metadata.

1. Base Module

The Base Module utilizes an XLNet-based cross-encoder to model the relationship between a mention's local context and the candidate entity's description. It also incorporates a "prior probability" score, representing the global popularity of an entity for a given mention string.

2. Auxiliary Data Module

The core innovation lies in the Auxiliary Data Module, which processes potentially voluminous and noisy metadata through a two-stage pipeline:

  • Selection: Using string similarity metrics (e.g., Levenshtein, Jaro-Winkler), the system identifies the top-$k$ most relevant auxiliary texts from the available pool. The study found that $k=3$ provides the optimal balance between signal and noise.
  • Encoding: A Longformer model jointly encodes the entity description alongside the selected auxiliary texts. Longformer's ability to process long sequences makes it ideal for handling multiple context snippets.
The dual-module architecture, featuring a Base Module for local context and an Auxiliary Data Module for processing external metadata.

The final entity ranking score is a fusion of the outputs from both modules, trained using cross-entropy loss.

Experimental Evaluation

To evaluate the framework, the authors introduced QuoraEL, a dataset containing over 8,000 labeled mentions across 504 CQA texts. The knowledge base was derived from a 2019 Wikipedia dump.

Key Results

  • State-of-the-Art Performance: The full framework reached 92.02% accuracy, outperforming established models like BLINK (87.92%) and Zeshel (88.91%) even when those models were provided with auxiliary data.
  • Ablation Insights: Removing auxiliary data led to a 2.65% drop in accuracy, confirming the value of external metadata. Parallel answers and topic tags were found to be particularly potent signals.
  • Model Agnosticism: The Auxiliary Data Module can be integrated into existing EL models. When added to the Deep-ED model, it yielded a significant 5.6% performance boost.

Conclusion

By effectively filtering and encoding auxiliary data, this research demonstrates that the "noise" of CQA platforms can be transformed into a powerful asset for entity disambiguation. While computationally more expensive than simple bi-encoders, the performance gains justify the use of cross-encoders and Longformers in this domain. Future work may involve extending this framework to more technical domains, such as software engineering forums.

Summary updated

Abstract

Community Question Answering (CQA) platforms contain plenty of CQA texts (i.e., questions and answers corresponding to the question) where named entities appear ubiquitously. In this paper, we define a new task of CQA entity linking (CQAEL) as linking the textual entity mentions detected from CQA texts with their corresponding entities in a knowledge base. This task can facilitate many downstream applications including expert finding and knowledge base enrichment. Traditional entity linking methods mainly focus on linking entities in news documents, and are suboptimal over this new task of CQAEL since they cannot effectively leverage various informative auxiliary data involved in the CQA platform to aid entity linking, such as parallel answers and two types of meta-data (i.e., topic tags and users). To remedy this crucial issue, we propose a novel transformer-based framework to effectively harness the knowledge delivered by different kinds of auxiliary data to promote the linking performance. We validate the superiority of our framework through extensive experiments over a newly released CQAEL data set against state-of-the-art entity linking methods.
BibTeX
@inproceedings{li2022community,
  author = {Yuhan Li and Wei Shen and Jianbo Gao and Yadong Wang},
  title = {{Community Question Answering Entity Linking via Leveraging Auxiliary Data}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2022},
  doi = {10.24963/ijcai.2022/298},
}
Figure