TLR at BSNLP2019: A Multilingual Named Entity Recognition System

Jose G. Moreno, Elvys Linhares Pontes, Mickael Coustaty, Antoine DoucetACL 2019

TL;DR

This paper presents a multilingual Named Entity Recognition (NER) system that leverages a combination of language-specific and multilingual contextual embeddings. Their approach achieves encouraging results across four languages, demonstrating the effectiveness of a unified architecture for cross-lingual NER tasks.

What Problem Does This Paper Address?

Named Entity Recognition (NER) is the task of identifying and categorizing named entities (like people, organizations, and locations) in text. While NER has seen significant advancements, extending these systems to handle multiple languages simultaneously, especially for news articles with diverse topics, remains a challenge. The BSNLP2019 shared task specifically aimed to push the boundaries of multilingual NER by requiring participants to build systems capable of recognizing entities in Bulgarian, Czech, Polish, and Russian. The authors tackle the problem of creating a single, robust NER system that can perform well across these different languages without extensive language-specific pre-processing.

Methods

The core of the TLR system is a standard neural architecture for sequence labeling, specifically the LSTM-CNNs-CRF model. What makes their approach novel is the integration of multilingual contextual embeddings from BERT. To manage the computational cost and complexity of using all BERT layers, they employed a weighted average strategy, focusing on the first two layers. This multilingual BERT embedding is combined with language-specific fastText embeddings. For the final prediction, they used a majority voting schema across four individual NER models, each trained with a specific language's fastText embeddings. Character-level information is also incorporated using a CNN, and case encoding is handled as a language-independent feature.

Datasets

The authors trained and evaluated their models on datasets provided for the BSNLP2019 shared task, which included two topics: "nord_stream" and "ryanair." They also utilized data from the previous BSNLP2017 edition for Bulgarian, Czech, Polish, and Russian. Additionally, they incorporated the standard CoNLL2003 dataset for English to perform comparative evaluations, noting that English was not a target language for the BSNLP task.

Key Insights and Ideas

  • Unified Architecture for Multilingual NER: The paper demonstrates that a single neural architecture, augmented with appropriate embeddings, can effectively handle NER across multiple languages, reducing the need for language-specific model development.
  • BERT's Multilingual Power: The use of multilingual BERT embeddings proved crucial for achieving good cross-lingual performance, highlighting its ability to capture contextual information across diverse languages.
  • Voting Schema for Robustness: While individual language-specific models had their strengths, the majority voting schema across these models ultimately provided better average performance, suggesting a robust way to combine predictions.
  • Language-Specific Embeddings Still Matter: Despite the success of multilingual BERT, the paper found that using language-specific fastText embeddings generally improved performance for individual language models when compared to using only multilingual BERT.

Key Results

The TLR system achieved encouraging results on the BSNLP2019 task, with "Relaxed Partial" metrics averaging 80.26% across the four languages. For the "nord_stream" topic, the "Relaxed Partial" metric reached up to 83.384% for Bulgarian. For the "ryanair" topic, the "Relaxed Partial" metric was as high as 83.419% for Russian. When evaluated on the CoNLL2003 English dataset, their system with two BERT layers achieved an F1 score of 89.0%, and with six layers, it reached 89.8%.

Limitations and Future Work

The authors acknowledge that their system's performance on English (a non-target language for BSNLP) was slightly lower than state-of-the-art, which they attribute to the focus on multilingual training and the use of only two BERT layers to manage computational resources. They also note that while their voting schema is effective, a system that is more explicitly aware of the input language might yield even better results. Future work includes exploring the use of more BERT layers and investigating more sophisticated methods for combining multilingual and language-specific features. They also plan to address entity normalization and linking in future iterations.

Summary updated

BibTeX
@inproceedings{moreno2019tlr,
  author = {Jose G. Moreno and Elvys Linhares Pontes and Mickael Coustaty and Antoine Doucet},
  title = {{TLR at BSNLP2019: A Multilingual Named Entity Recognition System}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2019},
}
Figure