Abstract
The MultiCoNER II task aims to detect complex, ambiguous, and fine-grained named entities in low-context situations and noisy scenarios like the presence of spelling mistakes and typos for multiple languages. The task poses significant challenges due to the scarcity of contextual information, the high granularity of the entities (up to 33 classes), and the interference of noisy data. To address these issues, our team PAI proposes a universal Named Entity Recognition (NER) system that integrates external entity information to improve performance. Specifically, our system retrieves entities with properties from the knowledge base (i.e. Wikipedia) for a given text, then concatenates entity information with the input sentence and feeds it into Transformer-based models. Finally, our system wins 2 first places, 4 second places, and 1 third place out of 13 tracks.
Introduction: The Challenge of Complex NER
Named Entity Recognition (NER) is a fundamental task in NLP, yet standard models frequently struggle when sentences are short, noisy, or contain highly ambiguous names. The MultiCoNER II challenge highlights these "complex" scenarios, focusing on low-context snippets where surrounding words are insufficient to determine an entity's type. Furthermore, the task requires distinguishing between 33 fine-grained classes (e.g., identifying if "Apple" refers to a Product, Creative Work, or Corporation) and handling synthetic noise like typos.
As illustrated in Figure 1, without external knowledge, a model is essentially guessing the category of an entity like "Bryan Talbot." By retrieving external metadata, the system can definitively identify him as an artist.
Proposed Methodology: Knowledge-Augmented Transformers
To bridge this information gap, Team PAI developed a system that effectively "consults" a knowledge base before making a classification decision. The architecture consists of several key components:
- Knowledge Retrieval Module: The team constructed an offline Knowledge Base (KB) containing over 90 million entities from Wikidata. For each entity, critical properties such as
instanceof,subclassof, andoccupationwere indexed. - Context Injection: During processing, the system performs string matching to identify potential entities in the input sentence. It then retrieves relevant properties and appends them to the original text using a specific format:
[Sentence] $ [Entity Name] | [Property 1] | [Property 2]. - Entity-Aware Attention: Standard Transformers utilize global attention, which can lead to confusion when multiple entities and their respective metadata are present. Team PAI introduced a custom attention mask that explicitly links tokens in the original sentence to their corresponding retrieved properties, preventing the model from being distracted by irrelevant metadata.
As shown in Figure 4, the model backbone (typically mBERT or Chinese BERT) processes the augmented input, guided by the entity-aware attention matrix to maintain semantic clarity between tokens and their external context.
Key Insights and Results
The integration of Wikidata properties proved to be a decisive factor in the competition. The team's research yielded several critical insights:
- Property Significance: Specific metadata fields act as strong signals for certain entity types. For example, the
occupationproperty is highly predictive forPersonentities, whilesubclassofis vital forProductclassification. Ablation studies showed that removing theoccupationproperty alone led to a 4.5 F1 point drop in performance. - Superior Coverage: The Wikidata-based "Property Dict" achieved a 92.8% coverage rate on development data, vastly outperforming traditional manual dictionaries which only reached 48.5%. This was particularly impactful for low-resource languages like Bangla and Hindi, where coverage exceeded 95%.
- Competitive Success: The PAI system was highly successful across the 13 multilingual tracks, securing 2 first places, 4 second places, and 1 third place. On average, the property-enhanced approach outperformed the baseline by 12.2 F1 points.
Conclusion
Team PAI's universal NER system demonstrates that external knowledge retrieval is a powerful solution for the low-context and high-ambiguity problems inherent in modern NER tasks. While the system is robust, future work will focus on incorporating fuzzy matching (e.g., via ElasticSearch) to better handle the retrieval of misspelled entities in noisy data environments.
Summary updated