Simplify-Then-Translate: Automatic Preprocessing for Black-Box Translation

Sneha Mehta, Bahareh Azarnoush, Boris Chen, Avneesh Saluja, Vinith Misra, Ballav Bihani, Ritwik KumarAAAI 2020 · DOI

Abstract

Commercial machine translation (MT) systems, while powerful, often struggle with low-resource languages and idiomatic expressions due to their "black-box" nature. This paper proposes a "Simplify-Then-Translate" framework that uses an Automated Preprocessing (APP) model to transform complex source sentences into literal, compositional forms. By training the APP model on "translationese" generated via back-translation from high-resource languages, the authors achieve significant improvements in translation quality across various low-resource target languages without modifying the underlying MT system.

Introduction

State-of-the-art machine translation (MT) systems like Google Translate or Microsoft Translator are frequently treated as black boxes. While they perform exceptionally well on high-resource language pairs, they often fail when encountering low-resource languages or domain-specific nuances, such as colloquialisms in TV subtitles. A common failure mode involves non-compositional phrases—idioms like "jump in" (meaning to intervene)—which are often translated literally, resulting in nonsensical output.

Since the internal parameters of these commercial systems are inaccessible, the authors explore an alternative: fixing the input. The proposed Automated Preprocessing (APP) model simplifies source text to make it more "digestible" for black-box MT systems.

The Simplify-Then-Translate Framework

The core of the framework is a Transformer-based sequence-to-sequence model trained to map "natural" English into "simplified" English. To generate training data without manual effort, the authors utilize a novel back-translation approach:

  1. Back-Translation: A high-resource parallel corpus (e.g., English-Spanish) is used. The Spanish side is translated back into English using a black-box MT system.
  2. Targeting "Translationese": The resulting back-translated English, while grammatically correct, tends to be more literal and less idiomatic than the original—a style often called "translationese."
  3. APP Training: The model learns to transform original English sentences into these simplified versions.
Comparison of literal MT error versus the improved translation achieved through simplification.

At inference time, the source sentence is first processed by the APP model. This simplified output is then fed into the black-box MT system for final translation into the target low-resource language.

Experimental Evaluation

The authors evaluated the framework using several datasets:

  • FIGS: 9.5 million subtitle events from Netflix (English paired with French, Italian, German, and Spanish) for in-domain training.
  • WikiLarge: A Wikipedia-based simplification benchmark for out-of-domain testing.
  • Target Languages: Harder or low-resource languages including Hindi (Hi), Bulgarian (Bg), Hungarian (Hu), and others.

Results and Analysis

The APP model provided consistent improvements across all tested languages. Relative BLEU scores increased by 3.5% to 11.6%, and the Translation Edit Rate (TER) decreased significantly, indicating reduced human post-editing effort.

GLEU score distribution for English-Hindi translation, showing a rightward shift (improvement) when using the revised/simplified input.

Key insights from the study include:

  • Domain Specificity: Simplifiers trained on Wikipedia data actually harmed subtitle translation, highlighting the need for domain-matched training data.
  • Idiom Reduction: The model effectively converts non-compositional phrases (e.g., "over your head") into compositional ones (e.g., "complicated"), which are less likely to be mistranslated.
  • Source Language Choice: Using "easier" high-resource pairs (like English-Spanish) for back-translation yielded better simplification models than "harder" pairs.

Conclusion and Future Work

The "Simplify-Then-Translate" approach offers a robust, model-agnostic method for improving black-box MT. While highly effective, the authors noted limitations such as occasional pronoun inconsistency. Future research may explore multi-reference simplification using multiple high-resource languages and extending the framework to non-English source languages.

Summary updated

BibTeX
@inproceedings{mehta2020simplify,
  author = {Sneha Mehta and Bahareh Azarnoush and Boris Chen and Avneesh Saluja and Vinith Misra and Ballav Bihani and Ritwik Kumar},
  title = {{Simplify-Then-Translate: Automatic Preprocessing for Black-Box Translation}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2020},
  doi = {10.1609/aaai.v34i05.6369},
}
Figure