Abstract
Commercial machine translation (MT) systems, while powerful, often struggle with low-resource languages and idiomatic expressions due to their "black-box" nature. This paper proposes a "Simplify-Then-Translate" framework that uses an Automated Preprocessing (APP) model to transform complex source sentences into literal, compositional forms. By training the APP model on "translationese" generated via back-translation from high-resource languages, the authors achieve significant improvements in translation quality across various low-resource target languages without modifying the underlying MT system.
Introduction
State-of-the-art machine translation (MT) systems like Google Translate or Microsoft Translator are frequently treated as black boxes. While they perform exceptionally well on high-resource language pairs, they often fail when encountering low-resource languages or domain-specific nuances, such as colloquialisms in TV subtitles. A common failure mode involves non-compositional phrases—idioms like "jump in" (meaning to intervene)—which are often translated literally, resulting in nonsensical output.
Since the internal parameters of these commercial systems are inaccessible, the authors explore an alternative: fixing the input. The proposed Automated Preprocessing (APP) model simplifies source text to make it more "digestible" for black-box MT systems.
The Simplify-Then-Translate Framework
The core of the framework is a Transformer-based sequence-to-sequence model trained to map "natural" English into "simplified" English. To generate training data without manual effort, the authors utilize a novel back-translation approach:
- Back-Translation: A high-resource parallel corpus (e.g., English-Spanish) is used. The Spanish side is translated back into English using a black-box MT system.
- Targeting "Translationese": The resulting back-translated English, while grammatically correct, tends to be more literal and less idiomatic than the original—a style often called "translationese."
- APP Training: The model learns to transform original English sentences into these simplified versions.
At inference time, the source sentence is first processed by the APP model. This simplified output is then fed into the black-box MT system for final translation into the target low-resource language.
Experimental Evaluation
The authors evaluated the framework using several datasets:
- FIGS: 9.5 million subtitle events from Netflix (English paired with French, Italian, German, and Spanish) for in-domain training.
- WikiLarge: A Wikipedia-based simplification benchmark for out-of-domain testing.
- Target Languages: Harder or low-resource languages including Hindi (Hi), Bulgarian (Bg), Hungarian (Hu), and others.
Results and Analysis
The APP model provided consistent improvements across all tested languages. Relative BLEU scores increased by 3.5% to 11.6%, and the Translation Edit Rate (TER) decreased significantly, indicating reduced human post-editing effort.
Key insights from the study include:
- Domain Specificity: Simplifiers trained on Wikipedia data actually harmed subtitle translation, highlighting the need for domain-matched training data.
- Idiom Reduction: The model effectively converts non-compositional phrases (e.g., "over your head") into compositional ones (e.g., "complicated"), which are less likely to be mistranslated.
- Source Language Choice: Using "easier" high-resource pairs (like English-Spanish) for back-translation yielded better simplification models than "harder" pairs.
Conclusion and Future Work
The "Simplify-Then-Translate" approach offers a robust, model-agnostic method for improving black-box MT. While highly effective, the authors noted limitations such as occasional pronoun inconsistency. Future research may explore multi-reference simplification using multiple high-resource languages and extending the framework to non-English source languages.
Summary updated