Abstract
Optimizing Neural Machine Translation (NMT) for inference speed is a standard industry practice, yet it introduces significant risks regarding model fairness. This study investigates how common speed-up techniques—including quantization, shallow decoders, and average attention networks—impact gender translation accuracy. Using a novel benchmark, SimpleGEN, we demonstrate that as models are compressed and accelerated, they increasingly rely on training-set stereotypes, leading to a disproportionate degradation in accuracy for sentences that challenge traditional gender roles.
Introduction
In production environments, NMT models must balance translation quality with low latency. Engineers typically optimize for speed using techniques like weight quantization or architectural pruning, measuring success via the BLEU metric. However, BLEU is a coarse measure of word overlap and often fails to capture nuanced linguistic errors, such as incorrect gender agreement.
This research explores whether optimized models "take shortcuts" by defaulting to stereotypical gender associations when computational resources are constrained. Crucially, we examine if these failures occur even when the source text contains explicit, unambiguous gender cues.
Methodology
We evaluated several industry-standard optimization techniques on Transformer-based models for English-to-Spanish (En-Es) and English-to-German (En-De) translation:
- Search Optimization: Reducing beam size to 1 (greedy search).
- Shallow Decoders (SD): Reducing the number of decoder layers.
- Average Attention Networks (AAN): Simplifying the self-attention mechanism in the decoder.
- Quantization (INT8): Compressing 32-bit weights to 8-bit integers.
These methods were tested both individually and in a "stacked" configuration to simulate a high-performance production environment.
The SimpleGEN Benchmark
To isolate gender bias, we introduced SimpleGEN, a template-based evaluation set. Unlike previous datasets that focus on ambiguous gender, SimpleGEN uses sentences where the gender of an occupation is explicitly defined by context (e.g., "That physician is a funny lady!").
The dataset categorizes sentences into:
- Pro-stereotypical: Aligning with traditional roles (e.g., male engineer).
- Anti-stereotypical: Challenging traditional roles (e.g., female mechanic).
Results and Analysis
The primary finding is a clear "decoupling" between general translation quality (BLEU) and gender accuracy. While optimizations like stacking all techniques resulted in an 80% reduction in decoding time with only a 6-9% relative drop in BLEU, the impact on gender accuracy was far more severe.
The Anti-Stereotype Penalty
Optimized models exhibit a "lazy" behavior, increasingly ignoring contextual clues in favor of training data frequencies. This is most evident in anti-stereotypical cases. For English-to-German, the accuracy for anti-stereotypical sentences dropped by 41.3%, significantly outpacing the decline in BLEU.
Vulnerable Architectures
Among the individual optimizations, Average Attention Networks (AANs) were identified as particularly high-risk, causing substantial degradation in gender accuracy despite providing significant speedups. Furthermore, sentences involving female contexts (FC) consistently saw larger accuracy drops than those involving male contexts across all tested languages.
Conclusion
The trade-off in NMT optimization is not merely between speed and quality, but between speed, quality, and bias. Relying solely on BLEU for production deployment can mask significant fairness regressions. We recommend that bias-specific benchmarks like SimpleGEN be integrated into standard NMT evaluation pipelines to ensure that performance gains do not come at the cost of amplified societal stereotypes.
Summary updated