Abstract
The GoURMET project, funded by the European Commission’s H2020 program (under grant agreement 825299), develops models for machine translation, in particular for low-resourced languages. Data, models and software releases as well as the GoURMET Translate Tool are made available as open source.
Introduction
While Neural Machine Translation (NMT) has reached near-human parity for high-resource language pairs like English-Spanish, it remains notoriously unreliable for "low-resourced" languages—those with limited digital data available for training. This gap creates a critical bottleneck for international news organizations such as the BBC and Deutsche Welle (DW). These organizations must monitor local media and produce content in dozens of languages where commercial MT tools often fail or produce "hallucinations"—fluent-sounding but factually incorrect text. The GoURMET project addresses this by creating robust, domain-specific NMT models and sustainable tools specifically for journalists and media analysts.
Methodology and Datasets
To overcome the lack of training data, the project utilized several advanced NMT strategies:
- Multi-Task Learning Data Augmentation (MTL DA): Researchers generated synthetic parallel sentences. Even when these were statistically "unlikely," they systematically improved quality and reduced hallucinations.
- Transfer Learning: Applied between topical domains (e.g., COVID-19) to maintain accuracy as news cycles shift.
- Rapid Deployment: The pipeline's agility was tested using Pashto as a "surprise language," developing a functional model in just two months.
The models were trained on a mix of public data and proprietary content from the BBC and DW, covering 16 languages including Amharic, Burmese, Gujarati, Hausa, Swahili, and Yoruba. Professional editors manually annotated bilingual datasets to ensure high-quality ground truth.
Key Insights: Human-Centric Robustness
A primary finding of the project is that for journalists, robustness is more valuable than mere fluency. A model that admits ignorance is preferable to one that "hallucinates" a plausible lie. Consequently, the project moved beyond automated metrics like BLEU scores, which can be misleading for low-resource languages. Instead, they employed "gap filling" and "post-editing" metrics to measure the actual effort saved by a human editor using the MT output.
Results and Integration
The GoURMET models are now integrated into production workflows. They power the BBC’s "Frank" multilingual prototype and DW’s "plain X" platform for automated translation and subtitling.
Technical benchmarking showed that GoURMET models are competitive with, and in some domain-specific cases superior to, major commercial services like Google MT. To ensure long-term sustainability, the project released the GoURMET Translate Tool and its models as open source.
Conclusion
While the project successfully developed models for 16 languages and demonstrated rapid deployment capabilities, challenges remain. The models are currently optimized for news and health domains rather than technical or creative literature. Future work will focus on maintaining a sustainable platform for media analysts and expanding the scope to include more of the hundreds of underserved low-resourced languages.
Summary updated