UoS Participation in the WMT20 Translation of Biomedical Abstracts

Felipe Soares, Delton VazWMT @ EMNLP 2020 · OpenReview

Abstract

Researchers from the University of Sheffield (UoS) developed a high-performing neural machine translation (NMT) system for the WMT20 Biomedical Translation task. By leveraging the Transformer architecture and a massive "data-hungry" approach—combining specialized medical corpora with general patent data—they achieved state-of-the-art results for Portuguese-English translation pairs. This paper outlines the methodology, including back-translation and incremental training, and analyzes the impact of data volume and preprocessing on translation quality across ten language directions.

Introduction

Translating biomedical abstracts requires navigating highly specific terminology and complex sentence structures. The annual Conference on Machine Translation (WMT) hosts a shared task to push the boundaries of NMT in this domain. The UoS team addressed ten translation directions (English to/from Spanish, Portuguese, Russian, Italian, and French), focusing on how to effectively train deep learning models when high-quality in-domain data is scarce compared to general-purpose text.

Methodology

The team utilized the Transformer architecture, implemented via Google’s TensorFlow Model Garden. Key technical strategies included:

  • Back-translation: Synthetic parallel data was created by translating English text back into the source languages to improve models where English was the target.
  • Incremental Training: For the English/Russian pair, which had limited medical data, the model was first trained on a large set of patent abstracts (out-of-domain) and then fine-tuned on medical datasets.
  • WordPiece Tokenization: An unsupervised tokenizer with a 32,000-word vocabulary helped the model handle rare medical terms by breaking them into sub-units.
  • Rigorous Filtering: To prevent data leakage, string-matching tools were used to remove any training sentences that appeared in the official MEDLINE test sets.

Data Resources

The team aggregated a diverse set of resources to create a massive training pool, ranging from 1.08 million sentence pairs for Italian to 6.49 million for Portuguese.

Table 1: Summary of corpora and sentence counts for each language pair.

The datasets included in-domain medical sources like SciELO, EMEA, and MEDLINE, as well as the ParaPat patent corpus, which provided a baseline for formal technical language.

Key Results and Analysis

The UoS system was a top performer in several categories, particularly in Portuguese.

Table 2: BLEU scores for the UoS system across ten translation directions.

As shown in Table 2, the team achieved the highest BLEU scores for both EN→PT (0.4744) and PT→EN (0.5334). Spanish-English results were also highly competitive. However, performance in French, Italian, and Russian lagged behind leaders due to encoding issues—specifically how the system handled apostrophes and certain Cyrillic characters—highlighting the critical importance of basic text-cleaning.

Discussion

The standout success in Portuguese suggests that for Transformer models, sheer data volume (6.49M sentences) can sometimes outweigh the need for perfectly curated in-domain data. Furthermore, the use of incremental training for Russian demonstrates a viable path for low-resource domains: starting with a related technical domain (patents) to teach the model technical language before narrowing down to medical specifics.

Conclusion

The UoS participation in WMT20 demonstrated the power of large-scale data aggregation and the Transformer architecture in biomedical translation. Future work will focus on refining data selection to identify the most relevant out-of-domain sentences and exploring specialized decoding techniques to better handle scientific terminology.

Reference: Soares, F., & Vaz, D. A. (2020). UoS Participation in the WMT20 Translation of Biomedical Abstracts. In Proceedings of the 5th Conference on Machine Translation (WMT), pages 870–874.

Summary updated

BibTeX
@inproceedings{soares2020uos,
  author = {Felipe Soares and Delton Vaz},
  title = {{UoS Participation in the WMT20 Translation of Biomedical Abstracts}},
  booktitle = {WMT @ EMNLP},
  year = {2020},
  url = {https://openreview.net/forum?id=dYxim67SrBy},
}
Figure