Entity Projection via Machine Translation for Cross-Lingual NER

Alankar Jain, Bhargavi Paranjape, Zachary C. LiptonEMNLP 2019 · DOI

TL;DR

This paper introduces a novel "Translate-Match-Project" (TMP) method for cross-lingual Named Entity Recognition (NER) that leverages machine translation (MT) to project annotations from resource-rich languages to those with limited annotated data. TMP achieves state-of-the-art results on several languages, outperforming existing methods by effectively handling the challenges of imperfect MT alignment.

What Problem Does This Paper Address?

Named Entity Recognition (NER) is crucial for information retrieval and question answering, but building high-quality NER models typically requires massive amounts of annotated data. This creates a significant imbalance, as many languages have extensive machine translation support but lack sufficient annotated corpora for NER. This paper addresses the challenge of building effective cross-lingual NER systems for "medium-resource" languages – those with good MT but limited NER data – by leveraging existing MT systems to transfer knowledge from high-resource languages.

Methods

The core of the proposed method is the "Translate-Match-Project" (TMP) pipeline, which involves three main steps:

  1. Translate: The annotated source language corpus (English in their experiments) is translated into the target language using an off-the-shelf MT system (Google Translate). This creates an unlabeled target dataset.
  2. Match: Named entities from the source sentences are projected onto the translated target sentences. This involves:
  3. Candidate Match Generation: For each source entity, potential translations are generated using MT and bilingual lexicons. These potential translations are then matched against target sentence tokens using a "soft affix-matching heuristic" that considers both orthographic and phonetic similarity.
  4. Span Match Generation: Token-level matches are grouped into spans to form multi-token target entities.
  5. Best Match Selection: The best matching source-target entity pairs are selected based on character edit distance.
  6. Distribution-based Matching: A final step leverages corpus-level consistency (using tf-idf scores) to resolve remaining discrepancies and match unmatched entities.
  7. Project: The NER tags from the source entities are projected onto their aligned target entities.

The novelty lies in the multi-stage matching process that accounts for imperfect MT alignment, including phonetic similarity, exact copy translations, and distribution-based matching, and the dual use of MT (once for sentences, once for entities).

Datasets

The paper uses English NER data from the CoNLL 2003 shared task as the source language. For target languages, they evaluate on:

  • European Languages: Spanish, Dutch, and German (using CoNLL 2002 and 2003 datasets).
  • Indo-Aryan Language: Hindi (using FIRE 2013 shared task data, with custom splits).
  • Dravidian Language: Tamil (using FIRE 2013 shared task data, with custom splits).
  • Simplified Chinese: Using OntoNotes 4.0 (2012) English data for sampling.
  • Medium-Resource Language: Armenian (using a newly introduced test corpus from Ghukasyan et al. (2018) with silver annotations from Wikipedia).

Key Insights and Ideas

  • MT as a Bridge for Low-Resource NER: The paper convincingly demonstrates that machine translation, even with its inherent imperfections, can be a powerful tool for bridging the gap in annotated data for NER across languages.
  • Beyond Word-for-Word Alignment: The core innovation is the sophisticated matching strategy that goes beyond simple word-level alignment. By incorporating phonetic similarity, handling direct translations, and using distributional statistics, the method robustly handles cases where MT produces non-literal or contextually different translations.
  • Dual Use of MT is Key: Using MT not only for sentence translation but also for generating candidate entity translations significantly improves the robustness of the projection process.
  • Phonetic and Lexical Cues Matter: The ablation study highlights the importance of phonetic matching (especially for morphologically rich languages like Spanish) and the use of gold lexicons (for Hindi) in improving alignment accuracy.
  • Generalizability to Medium-Resource Languages: The successful application to Armenian, a language with limited prior NER resources, showcases the practical utility of the approach for rapid deployment.

Key Results

  • The proposed TMP method achieves state-of-the-art results on cross-lingual NER for Spanish (+1.1 F₁), German (+1.4 F₁), and Chinese (+5 F₁), and beats baselines on Hindi (+2.1 F₁) and Tamil (+5 F₁).
  • For Armenian, TMP achieves an F₁ score of 62.6, outperforming even a monolingual model trained on Armenian source data by 0.4 points.
  • The method shows marked improvements over word-by-word translation baselines (like BWET) on languages with significantly different word orders from English (e.g., German, Hindi, Tamil, Chinese).
  • Qualitative analysis shows that TMP produces fewer noisy tags and better alignment compared to the fast-align baseline.

Limitations and Future Work

  • MT System Dependency: The approach is limited to languages supported by off-the-shelf MT systems.
  • Distribution Shift: The performance can be affected by distribution shift between older source language data (like CoNLL 2003) and newer target language data.
  • Monolingual Model Performance: While TMP achieves state-of-the-art cross-lingual results, it still falls short of models trained on large monolingual corpora.

Future work suggested includes:

  • Extending the method to languages not supported by Google Translate using pivot languages.
  • Incorporating domain adaptation techniques to mitigate distribution shift.
  • Further improving MT quality and exploring alternative MT systems.

Summary updated

BibTeX
@inproceedings{jain2019entity,
  author = {Alankar Jain and Bhargavi Paranjape and Zachary C. Lipton},
  title = {{Entity Projection via Machine Translation for Cross-Lingual NER}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2019},
  doi = {10.18653/v1/d19-1100},
}
Figure