AlphaMWE: Construction of Multilingual Parallel Corpora with MWE Annotations

Lifeng Han, Gareth Jones, Alan SmeatonCOLING 2020

TL;DR

This paper introduces AlphaMWE, a novel multilingual parallel corpus annotated with Multiword Expressions (MWEs) across English, Chinese, German, and Polish. By meticulously translating and annotating MWEs, the authors provide a valuable resource to improve machine translation (MT) systems, particularly in handling idiomatic and nuanced language.

What Problem Does This Paper Address?

Machine translation systems often struggle with Multiword Expressions (MWEs) – phrases where the meaning isn't simply the sum of its parts (e.g., "kick the bucket"). These expressions, including idioms, phrasal verbs, and other fixed phrases, pose a significant challenge because their translation requires understanding beyond literal word-for-word conversion. While monolingual MWE corpora exist, there's a critical lack of multilingual parallel corpora with MWE annotations. This gap hinders research in cross-lingual and multilingual NLP tasks, especially machine translation, which needs to accurately capture these nuanced expressions across languages.

Methods

The authors constructed the AlphaMWE corpus by leveraging the PARSEME shared task's English corpus as a starting point. They then employed state-of-the-art machine translation models (Microsoft Bing, GoogleMT, Baidu Fanyi, and DeepL) to translate these sentences into Chinese, German, and Polish. Crucially, human annotators then post-edited these translations, ensuring fluency and adequacy while specifically annotating the target-side MWEs. A rigorous quality control process, including cross-validation by multiple annotators, was implemented to minimize errors. The paper also includes a detailed analysis and categorization of MT errors related to MWE translation, offering insights into the specific challenges faced by current MT systems.

Datasets

The primary dataset used is the PARSEME shared task's English corpus, which contains MWE annotations. The authors then created the AlphaMWE corpus, a new multilingual parallel corpus with MWE annotations, covering English, Chinese, German, and Polish. This corpus is made publicly available.

Key Insights and Ideas

  • MWEs are a significant hurdle for MT: The study highlights that even advanced MT systems struggle to accurately translate MWEs, often resorting to literal translations or producing awkward phrasing.
  • Multilingual MWE data is scarce but vital: The lack of parallel corpora with MWE annotations is a major bottleneck for developing robust cross-lingual NLP applications.
  • Human annotation is key for quality: While MT can provide a starting point, human post-editing and annotation are essential for creating high-quality MWE-annotated parallel data.
  • MT errors are diverse: The paper categorizes MT errors related to MWEs into types like "common sense," "super sense," "abstract phrase," "idiom," "metaphor," and "ambiguity," providing a framework for understanding and addressing these issues.
  • Context matters immensely: Many MWE translation errors stem from MT systems' inability to leverage broader context (sentence, paragraph, or even social/literary knowledge) to disambiguate meaning.

Key Results

The paper presents the AlphaMWE corpus, containing 750 English sentences with MWE annotations, translated and annotated into Chinese, German, and Polish. While specific quantitative performance metrics of MT systems on this corpus are not the primary focus, the paper provides extensive qualitative analysis and examples of MT errors across various MWE categories, illustrating the challenges faced by current MT models. The authors demonstrate that DeepL generally performed better than other tested MT models, but still produced errors.

Limitations and Future Work

The authors acknowledge that the corpus is still relatively small (750 sentences) and that the MWE categories are primarily focused on verbal MWEs (vMWEs) as defined by PARSEME. Future work includes expanding the corpus to include more languages (e.g., Spanish, French, Italian) and other MWE genres. They also plan to further refine the MT error categorization and investigate how to incorporate broader context inference into MT models, potentially by leveraging web search results.

Summary updated

BibTeX
@inproceedings{han2020alphamwe,
  author = {Lifeng Han and Gareth Jones and Alan Smeaton},
  title = {{AlphaMWE: Construction of Multilingual Parallel Corpora with MWE Annotations}},
  booktitle = {International Conference on Computational Linguistics},
  year = {2020},
}
Figure