USTC-NELSLIP at SemEval-2023 Task 2: Statistical Construction and Dual Adaptation of Gazetteer for Multilingual Complex NER

Jun-Yu Ma, Jia-Chen Gu, Jiajun Qi, Zhenhua Ling, Quan Liu, Xiaoyi ZhaoSemEval @ ACL 2023 · DOI

Abstract

Complex Named Entity Recognition (NER) presents significant challenges due to fine-grained taxonomies, low-context environments, and multilingual requirements. This paper describes the USTC-NELSLIP system for SemEval-2023 Task 2 (MultiCoNER II). We propose SCDAG (Statistical Construction and Dual Adaptation of Gazetteer), a framework that enhances NER performance by statistically mapping external knowledge from Wikidata and employing a dual adaptation strategy to align gazetteer features with pre-trained language model representations. Our approach achieved first place in the Hindi track and demonstrated consistent improvements across 12 languages.

Introduction

Traditional NER systems often struggle with "Complex NER," where entities are semantically ambiguous or appear in short, low-context snippets like search queries. The MultiCoNER II task highlights three primary difficulties: a fine-grained taxonomy of 33 sub-categories, the prevalence of short queries with minimal context, and the need for robust performance across diverse languages. While gazetteers can provide valuable external knowledge, their effectiveness is often limited by low coverage and the "representation gap" between discrete gazetteer tags and continuous language model (LM) embeddings.

Methodology

The SCDAG framework introduces two primary innovations to bridge these gaps:

Statistical Gazetteer Construction

To overcome the limitations of manual mapping, we utilize a statistical approach to construct the gazetteer. By analyzing the frequency of Wikidata entities associated with specific labels in the training set, we map Wikidata types to the top two most frequent labels. This method significantly increases gazetteer coverage; for instance, English coverage improved from approximately 32% to 36%.

Dual Adaptation

The core of SCDAG is the dual adaptation strategy, which aligns the gazetteer network with the pre-trained XLM-RoBERTa model.

The SCDAG architecture featuring statistical gazetteer construction and dual adaptation.

As shown in Figure 1, the process involves two stages:

  1. Alignment Stage: A BiLSTM-based gazetteer network is trained to mimic the output distributions of the XLM-RoBERTa model by minimizing the KL Divergence at both sentence and entity levels. This ensures the gazetteer features reside in a compatible semantic space.
  2. Supervised Learning Stage: Both networks are jointly fine-tuned on the NER task. The final system employs an ensemble of three classifier heads: Softmax, CRF, and Span-based predictors.

Experimental Results

The system was evaluated on the MultiCoNER II dataset across 12 languages. To address the low-context nature of search queries, we augmented the training data with synthetic "Data-query" sets derived from MS-MARCO and ORCAS templates.

Performance comparison of SCDAG against baseline and integration methods across 12 languages.

As illustrated in Figure 6, the SCDAG framework consistently outperforms both the "Base" and "Integration" (simple concatenation) methods across all languages. The system was particularly successful in the Hindi track, securing the top position. Ablation studies confirmed the importance of both the dual adaptation mechanism and synthetic data; for example, removing synthetic data caused the English F1 score to drop from 0.721 to 0.682.

Conclusion

The USTC-NELSLIP team's success at SemEval-2023 underscores the importance of high-coverage external knowledge and effective representation alignment in Complex NER. By statistically constructing gazetteers and employing dual adaptation, SCDAG effectively mitigates the challenges of fine-grained classification and low-context scenarios in a multilingual setting. Future work will focus on denoising statistical gazetteers and incorporating richer semantic information from Wikipedia.

Summary updated

BibTeX
@inproceedings{ma2023ustc,
  author = {Jun-Yu Ma and Jia-Chen Gu and Jiajun Qi and Zhenhua Ling and Quan Liu and Xiaoyi Zhao},
  title = {{USTC-NELSLIP at SemEval-2023 Task 2: Statistical Construction and Dual Adaptation of Gazetteer for Multilingual Complex NER}},
  booktitle = {SemEval @ ACL},
  year = {2023},
  doi = {10.18653/v1/2023.semeval-1.89},
}

Loading related content...

Figure