NLPeople at SemEval-2023 Task 2: A Staged Approach for Multilingual Named Entity Recognition

Mohab Elkaref, Nathan Herr, Shinnosuke Tanaka, Geeth De MelACL 2023

TL;DR

This paper tackles the challenge of identifying and classifying complex, ambiguous named entities in low-context multilingual text. They propose a novel two-stage approach: first extracting potential entity spans, then classifying their types. This staged method significantly improves performance, especially in noisy data, leading to a 3rd place finish in the multilingual track of SemEval-2023 Task 2.

What Problem Does This Paper Address?

Named Entity Recognition (NER) is a fundamental task in Natural Language Processing (NLP), aiming to locate and categorize named entities (like people, organizations, locations) in text. However, real-world text often presents challenges: entities can be ambiguous (e.g., "Apple" as a company or a fruit), have fine-grained types (e.g., distinguishing between different types of organizations), and appear in low-context settings where surrounding information is scarce. The MultiCoNER shared task specifically focuses on these difficulties, particularly in a multilingual context where code-switching and language-specific nuances add further complexity. Previous approaches often relied on external knowledge bases, but these can struggle with entities not present in their knowledge or with noisy data.

Methods

The core innovation of this paper is a staged approach to NER, breaking down the task into two distinct phases:

  1. Span Extraction: This stage focuses solely on identifying the boundaries of potential named entities within a sentence. The authors fine-tune a pre-trained transformer model (XLM-Roberta) to classify tokens as either the beginning of an entity (B), inside an entity (I), or outside any entity (O). This stage is designed to be less sensitive to the semantic ambiguity or existence of entities in external knowledge, as it primarily relies on linguistic patterns.
  2. Entity Classification: Once potential entity spans are identified, this stage assigns a specific type to each span. This involves encoding the span and its corresponding entity type using contextual embeddings and then classifying it.

To enhance performance, especially in the classification stage, the authors incorporate additional context from Wikipedia. This context is retrieved in various formats, including:

  • Matched Titles: Exact matches of article titles from Wikipedia.
  • Linked Page Titles: Titles of pages linked within relevant Wikipedia paragraphs.
  • Paragraphs: Summary paragraphs of articles whose titles match.
  • Infoboxes: Structured information (like "Born," "Citizenship") from Wikipedia infoboxes.

They also experiment with combining these context types and use an ensemble of models trained with different context configurations to achieve their final results.

Datasets

The primary dataset used is the MultiCoNER v2 dataset, which is a multilingual dataset designed for fine-grained and noisy Named Entity Recognition. This dataset was introduced for the SemEval-2023 Task 2. The authors also leverage Wikipedia dumps for various languages to extract additional context.

Key Insights and Ideas

  • Decomposition is Key: Separating span extraction from entity classification proves highly effective. Span extraction, being more tied to linguistic structure, is more robust to noise and low-context settings.
  • Contextual Enrichment for Classification: While span extraction benefits from a staged approach, entity classification heavily relies on rich external context, particularly structured information like infoboxes and descriptive paragraphs.
  • Multilingualism and Code-Switching: The authors found that matching article titles from other languages was beneficial, highlighting the importance of handling code-switching in multilingual NER.
  • Noise Resilience: The staged approach shows a significant advantage in handling noisy data, with span extraction performance dropping less drastically compared to entity classification.

Key Results

  • The proposed staged approach achieved 3rd place on the multilingual track and an average of 6th place overall in the SemEval-2023 Task 2.
  • The Span Extraction stage achieved unlabelled F1 scores above 90% on 11 out of 13 languages on the clean dataset.
  • The Entity Classification stage, when using the pooled models, outperformed the stronger RaNER baseline on 7 out of 12 languages.
  • Paragraph-based context generally yielded the best performance for entity classification, followed by Infoboxes.

Limitations and Future Work

  • Sensitivity to Noise: While the staged approach is more resilient, the Entity Classification stage remains sensitive to noise in the data, indicating that further improvements in noise handling for classification are needed.
  • Computational Constraints: The authors acknowledge that they experimented with a limited set of context combinations due to time and computational constraints, suggesting that exploring more complex combinations could yield further improvements.
  • Future Work: The paper implicitly suggests that further research into robust context retrieval and integration, especially for noisy or low-resource scenarios, would be beneficial. Exploring more advanced ensemble techniques or different model architectures for the classification stage could also be promising.

Summary updated

BibTeX
@inproceedings{elkaref2023nlpeople,
  author = {Mohab Elkaref and Nathan Herr and Shinnosuke Tanaka and Geeth De Mel},
  title = {{NLPeople at SemEval-2023 Task 2: A Staged Approach for Multilingual Named Entity Recognition}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2023},
}
Figure