Filling Gender & Number Gaps in Neural Machine Translation with Black-box Context Injection

Amit Moryossef, Roee Aharoni, Yoav Goldberg

Translating the English sentence "I am a doctor" into Hebrew using standard tools like Google Translate typically yields the masculine form (ani rofe). Conversely, "I am a nurse" often results in the feminine (ani achot). This phenomenon is not a linguistic requirement but a statistical "guess" based on gender stereotypes prevalent in training data.

In the paper "Filling Gender & Number Gaps in Neural Machine Translation with Black-box Context Injection," researchers from Bar Ilan University and the Allen Institute for AI propose a surprisingly simple "black-box" method to fix this without retraining the underlying models or accessing their internal code.

The Problem: Morphological Ambiguity

Machine translation often involves moving from a "morphologically poor" language (like English) to a "morphologically rich" one (like Hebrew, Spanish, or Russian). While English verbs often remain the same regardless of the speaker's gender, target languages frequently require specific inflections to reflect the gender and number of the subject or audience.

Current Neural Machine Translation (NMT) systems are forced to commit to a gender even when the source text is ambiguous. Without extra context, they fall back on statistical biases. Previous solutions required retraining models with special "gender tags," which is expensive and impossible for proprietary systems.

Methodology: Black-Box Context Injection

The authors propose a wrapper method that treats the NMT system as a black box. Their approach relies on the observation that modern NMT systems are already capable of tracking "coreference chains" (linking pronouns like "she" to "I" within a context).

  1. Injection: They prefix the source sentence with a short "hint" that clarifies the speaker and audience. For example, to translate "I love you" where the speaker is female and the listener is male, the input becomes: "She said to him: I love you".
  2. Translation: The NMT system utilizes its internal attention mechanisms to correctly infer that the "I" in the sentence is feminine based on the artificial antecedent "She."
  3. Extraction: By using a specific paratactic structure, the system almost always produces a translated prefix that is easy to identify and strip away, leaving only the correctly inflected target sentence.
Figure 6: Examples of prefix-based context injection and its effect on morphological inflection across various languages.

Key Results and Insights

The method was evaluated using the Silverman Dataset (a stand-up monologue) and the VHW (2018) Corpus.

  • Accuracy Boost: On the English-to-Hebrew task, providing the correct gender/number hints improved translation quality by up to 2.3 BLEU points over the baseline Google Translate output.
  • Outperforming Retrained Models: On the English-French benchmark, this black-box method outperformed previous state-of-the-art models that had been specifically trained with gender tags.
  • Register Signaling: Interestingly, even when the prefix provided the "wrong" gender, BLEU scores sometimes increased. The authors hypothesize that adding "She said:" signals a conversational register to the model, which better matches the style of the test data than default formal translations.
Figure 4: BLEU score improvements on the Silverman dataset categorized by the specific speaker and audience prefixes used.

Cross-Linguistic Success and Limitations

The method successfully controlled gender and number in 6 out of 10 tested languages, including Hebrew, Spanish, French, Italian, Russian, and Romanian. However, it failed in 4 cases (Portuguese, Czech, Catalan, and Polish) where the NMT system either ignored the prefix or failed to produce a strippable output.

While the method requires knowing the "metadata" (who is speaking to whom) beforehand, it offers a powerful, zero-shot way to control the output of models users do not own. Future work could involve automatically inferring this context from document-level cues to make the injection process fully autonomous.

Summary updated

BibTeX
@misc{moryossef2019filling,
  author = {Amit Moryossef and Roee Aharoni and Yoav Goldberg},
  title = {{Filling Gender \& Number Gaps in Neural Machine Translation with Black-box Context Injection}},
  year = {2019},
  eprint = {1903.03467},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1903.03467},
}
Figure