Occupation Blinding for Trigger-Faithful Gender Translation

Machine translation systems often get gender wrong even when the English sentence states it plainly. Given "The doctor is a mother," a system translating into Spanish will often write the masculine el médico. It does this because its sense of "doctors are male" outweighs the word "mother" sitting in the same sentence. We propose to fix this with a trick borrowed from relation extraction: entity masking, which means hiding the specific name or noun behind a placeholder so the model must reason from the surrounding words. During fine-tuning only, we replace the occupation noun with a placeholder on both sides of some training pairs. The target placeholder carries a gender tag, and the agreeing articles and adjectives are left intact. The model then learns to produce gender agreement from the cue words ("she," "mother," "father") alone. At test time nothing is masked, so there is no extra pipeline and no broken grammar. The bet is that the habit of "reading gender off the cue" transfers to unmasked sentences and weakens the occupation stereotype.

This matters because the failure is not only a problem of missing information. Investigating Failures of Automatic Translation in the Case of Unambiguous Gender shows that systems as large as M2M-100 fail to reach even 70% accuracy on sentences where gender is fully determined by a cue word, and that feminine cues fare much worse than masculine ones across 20 languages. Most existing fixes either target ambiguous sentences or work on hidden representations. None removes the specific shortcut, which is the identity of the occupation word, from the gender decision while keeping that word available for translation.

Core Claim

A model should decide target gender from the gender cue, not from which occupation happens to appear. We claim a targeted intervention does this better than simply fine-tuning on the same data. The intervention trains on a mixture of normal sentence pairs and "blinded" pairs. In a blinded pair, the English occupation becomes [PER_1]. The aligned target noun becomes a gender-typed slot such as [PER_1_F] or [PER_1_M], while its determiner, adjectives, and participles keep their original agreement. The distinctive prediction is about generalization to occupations the model never saw in anti-stereotypical contexts during fine-tuning. Ordinary fine-tuning can memorize per-occupation corrections. Blinded training cannot, because in blinded examples the occupation is simply absent. So the advantage should be largest on held-out occupations, and it should show up as lower sensitivity of the gender decision to swapping one occupation for another.

How This Relates to Prior Work

Entity masking supplies the mechanism. Teaching Machines to Read and Comprehend anonymized entities so that reading-comprehension models could not answer from world knowledge. The file also describes zero-coverage mention randomization, where every entity mention is replaced by a unique random string so a model cannot associate names with types. Both apply masking to classification or reading tasks, not to generation.

The main constraint comes from Extract and Attend. It shows that hard-replacing entities with placeholders at inference produces target sentences whose inflections do not agree, and it resolves this with soft decoder prefixes at a latency cost of 389 ms rising to 624 ms. We accept this negative result and design around it: masking happens only during training, and the target keeps its agreement morphology around the slot.

The closest work at the intersection uses masking for other purposes. Automatically Identifying Gender Issues in Machine Translation using Perturbations masks occupations and refills them with a masked language model to find sentences whose translated gender flips. That is purely diagnostic. We reuse its perturbation idea as a measurement of how much the gender decision depends on the occupation.

GenWriter generalizes entities into placeholders to retrieve counter-gender templates for rewriting English biographies. It is monolingual, and the thing it debiases is a classifier. Generating Gender Alternatives in Machine Translation puts gender placeholder structures into translation output. It does so only for ambiguous entities and through a multi-model pipeline, and it explicitly leaves the unambiguous-override failure untouched.

Two mitigation lines are the competitors we must beat:

Finally, Does Context Help Mitigate Gender Bias in Neural Machine Translation? warns that added context can raise feminine accuracy through a "stereotype boost." We must therefore report anti-stereotypical and held-out-occupation accuracy separately, not only averages.

Research Program

Building the training pool. We mine English–target sentence pairs from Europarl, WMT, and IWSLT for German, Spanish, French, Italian, Russian, Ukrainian, Hebrew, and Arabic, the eight WinoMT languages. We keep pairs in which an English human-referring common noun is linked to an explicit gender cue: a gendered pronoun resolved by an off-the-shelf coreference system, or a kinship or gendered noun in an apposition or copula. We then:

  1. align the noun to the target with fast_align, the aligner WinoMT itself uses;
  2. read the target noun's gender with a morphological analyzer;
  3. drop nouns with fixed grammatical gender, such as French victime, whose gender does not track the referent, following the confound noted by the perturbation paper.

Because these corpora are heavily masculine (feminine terms are under 20% of the data per the context study, and Europarl contains only 30% female utterances per Gender Bias in Machine Translation), we rebalance the pool by cue gender.

Creating blinded pairs. For each pair, with probability $p$, we produce a blinded version. The source noun becomes an indexed untyped slot. The target noun's stem becomes a slot typed by the analyzer's gender, and everything else is left intact, so the model is still trained to produce agreeing determiners, adjectives, and participles. Sentences with two humans get [PER_1] and [PER_2], so the model must use coreference to decide which slot the cue governs.

Fine-tuning. We fine-tune NLLB-200 (600M distilled) and M2M-100 (418M) on the mixed pool. The ratio $p$ is swept over a small grid.

Held-out occupations. We partition occupations into a fine-tuning set and a held-out set, disjoint from the lexicon of each test benchmark where possible. This lets us separate learning a rule from memorizing corrections for specific occupations.

Ablations. Three variants isolate the key ingredients:

  • the typed target slot is replaced by an untyped slot, removing the direct gender supervision;
  • the placeholder is replaced by a random pseudoword on both sides, in the style of mention randomization, which removes the noun's identity while keeping the input looking like natural text;
  • the source is blinded but the target keeps the real noun, which tests whether hiding information only on the source side is enough.

Baselines

Do-nothing reference. The unmodified pretrained NLLB-200 and M2M-100 models. For the unambiguous benchmark, the reported reference is that M2M-100-scale systems "fail to achieve even 70% accuracy" (Investigating Failures of Automatic Translation in the Case of Unambiguous Gender). On WinoMT, Evaluating Gender Bias in Machine Translation reports systems near or below 50% accuracy for many languages, and a French industrial system at 80% on stereotypical roles versus 54% on non-stereotypical ones.

Plain fine-tuning (the most important baseline). We fine-tune on exactly the same rebalanced sentence pairs with $p = 0$: the same steps, learning-rate grid, and number of tuning trials. This is the strongest competitor on our specific question. Any gain from blinding must come from the masking, not from having seen more cue-bearing, gender-balanced data.

Counterfactual gender swapping. We add, for each training pair, a copy with the English cue and the target morphology flipped, generated via the same analyzer and filtered to grammatical outputs. This doubles the anti-stereotypical signal without hiding the occupation. We give it the same total number of training examples as our method.

GFST. We reimplement GFST on the same English source pool, with forward translation followed by morphological filtering. Its reported WinoMT results are:

  • average accuracy rising from 52.4% to 58.8% across five pairs;
  • English–German rising from 75.5% to 85.4%.

Adversarial learning. The adversarial method reports WinoMT accuracy rising from about 53% to as high as 78.9%, with the male–female gap $\Delta_G$ dropping 91% for English–French and 86% for English–German (Mitigating Gender Bias in Machine Translation through Adversarial Learning). We rerun it on our backbone where feasible, and otherwise cite these figures as context rather than as a matched comparison.

What counts as beating them. Our method must:

  • exceed plain fine-tuning on held-out-occupation, anti-stereotypical accuracy by a margin larger than the variation across three random seeds;
  • exceed GFST's reported 85.4% on English–German WinoMT when run on the same backbone family;
  • stay within about one COMET point of plain fine-tuning on WMT newstest.

The three ablations described above are run under the identical budget.

Why not just…?

Why not just fine-tune on gender-balanced, cue-bearing data? This is the real threat. It may raise accuracy just as much on the occupations it trains on. The blinding hypothesis predicts a gap only in two places: on held-out occupations, and in the occupation-swap flip rate. If plain fine-tuning matches us there too, the masking adds nothing and we will report that. The matched $p = 0$ run exists to settle exactly this.

Why not just use counterfactual gender swapping? Swapping balances which gender co-occurs with each occupation. It still lets the model condition its gender decision on the occupation's identity, which Investigating Failures of Automatic Translation in the Case of Unambiguous Gender identifies as the source of cue overrides. Swapping is also constrained by the analyzer's ability to regenerate target morphology correctly, the same bottleneck behind GFST discarding up to 98% of candidates. Blinding needs the analyzer only to label the gender of one aligned noun. We will nonetheless run swapping at equal data size, and the combination of swapping plus blinding as well.

Why not just add context or prompt a large model? The context study found that context-aware models raise English–German feminine generative accuracy on MT-GenEval from about 5% to 23%, but partly through a stereotype boost. MiTTenS shows that even the strongest foundation models it tested keep a worst-case accuracy of only 71.4%. Neither approach removes the occupation shortcut. We include a prompted large-model reference on the test sets for context, but it is not the matched comparison.

Why not just mask at inference, as the perturbation paper and hard-replacement entity translation do? Extract and Attend shows that hard replacement at inference breaks target agreement and adds latency. Our design masks only during training for this reason.

Proposed Experiments

Test sets.

  • WinoMT (3,888 sentences, 8 languages), scored by accuracy, $\Delta_G$, and the stereotype gap $\Delta_S$ between pro- and anti-stereotypical role assignments.
  • The unambiguous-gender benchmark of Renduchintala and Williams on the overlapping languages, reported separately for feminine and masculine cues to track the "feminine penalty."
  • MT-GenEval for English–German and other available pairs, using its generative feminine accuracy.
  • WMT newstest scored by COMET, to measure general quality cost.

MiTTenS translates into English, the opposite direction, so it serves only as a secondary check if we train reverse-direction models.

First decisive check: held-out occupations. Compare anti-stereotypical accuracy on held-out occupations for blinded training versus plain fine-tuning. The hypothesis fails if the gap is within seed variance.

Second decisive check: sensitivity to the occupation. Following the perturbation paper, we keep the cue fixed and use a masked language model to substitute occupations into natural sentences. We then measure how often the translated gender flips. Blinding should cut this flip rate relative to all baselines at matched overall accuracy. If accuracy rises but the flip rate does not fall, the gain is not coming from removing the shortcut.

Third check: ingredient ablations. The typed-versus-untyped slot and slot-versus-pseudoword ablations show whether the gender label on the slot is doing the work, or whether merely removing the noun's identity is enough.

Fourth check: grammar is not harmed. On unmasked test input, we measure agreement errors between each occupation noun and its determiners using the analyzer. We also report results separately for Russian and Ukrainian, where the adversarial method struggled on the similarly complex Czech. This confirms that training-time masking did not introduce the agreement breakage that hard replacement causes.