Multilingual translation models sometimes understand a sentence perfectly well and then answer in the wrong language. Asked to translate German into French, a model may produce English, or simply copy the German. This failure is called off-target translation. Second-language speech research has a name for a strikingly similar failure in people: learners who can hear a foreign sound contrast as well as native speakers but cannot produce it. This note asks whether a translation model, at the moment it drifts into the wrong language, still "perceives" that its own output is off-target. If it does, that signal already sits in the model's hidden states, and the model can be steered back onto the target language without the external language detectors and repeated forward passes that current fixes rely on.
The proposal has two parts. The first measures, for each zero-shot direction (a language pair never seen together in training), how well the model can recognize language identity and how well it can produce the right language, using the correlational and asymmetry designs of human perception–production studies. The second uses whatever self-perception survives as a closed-loop monitor. A light probe reads the model's own decoder state at each step. When it detects drift, a steering vector is added to the final layers, and only then. If the diagnosis is right, this should match the best decoding-time fixes on off-target rate at near-baseline cost. If the diagnosis is wrong, and the model's perception degrades along with its production, the monitor has nothing to read and the method should fail. Both outcomes are informative.
Core Claim
Off-target translation is a dissociation between perception and production: the model's internal representation of which language its prefix is in, and which language it should be in, stays accurate while the language it actually emits goes wrong. This dissociation can be measured per direction. Where it holds, the model's own perception is a cheaper and earlier drift detector than an external language-identification model applied to the decoded string.
Two lines of evidence motivate the claim. On the human side, Urdu speakers learning Mandarin matched native listeners in categorizing stops along a voice-onset-time continuum. Voice onset time is the delay between releasing a consonant and the vocal cords starting to vibrate. These same speakers produced shifted and overlapping onset times (Du et al.). Russian learners of Estonian could discriminate the vowels /ö/ and /õ/ yet reversed their front–back placement when speaking (Meister and Meister).
On the model side, logit-lens analysis, which decodes intermediate layers directly into the vocabulary, shows that multilingual language models resolve the correct concept in middle layers and fail only in the last few layers, where the output language is chosen (Wang et al.). Wang et al. study factual recall rather than drift during translation, and they intervene by bypassing the final layers entirely. We instead ask whether language identity itself remains readable during generation, and we intervene only when it signals trouble.
The human literature also constrains the claim, because perception does not always lead. Japanese learners produced English flaps before they could discriminate them (Yoneyama et al.). Elsewhere, perception and production correlate only moderately, with r ≈ .40–.70 in Kluge et al.. We therefore expect a mix of directions: some where perception is intact and production fails, which the monitor can fix, and some where both fail, which it cannot. Predicting in advance which directions fall where is part of the contribution.
Research Program
Measuring production. Production is measured as the off-target rate (OTR), the fraction of outputs a language identifier assigns to the wrong language. We compute it per zero-shot direction.
Measuring representational perception. We train linear probes on decoder hidden states at every layer and every decoding step. Each probe predicts the language of the prefix generated so far. Probes are trained only on supervised, on-target directions, so any accuracy on zero-shot outputs reflects transfer rather than memorization. We then ask a sharper question on off-target outputs. If we run the model's own wrong-language output back through it, does the probe still recognize which language that output is in? And does a second probe, trained to read the requested target language from the decoder state, still recover the target? This is the computational analogue of an identification task.
Measuring behavioral perception. For the instruction-following model (LLaMA 2), we ask it directly to name the language of its own off-target outputs. We also run an odd-one-out triad test, modelled on the categorical discrimination test of Kluge et al.. The model sees three sentences, two in the target language and one in the dominant language, and must identify the one that differs.
Analysis. Per direction, we correlate perception scores with OTR, as the human studies do. We classify each direction as perception-intact/production-failed, both-failed, or the reverse. We test whether "boundary width" predicts how vulnerable a direction is to the beam-search curse documented by Yang et al.. Boundary width here is the probe's margin between the target language and the dominant language, borrowed from the wider, fuzzier category boundaries that non-native listeners show in Meister and Meister. The beam-search curse is the finding that wider beams increase off-target output.
The self-monitor. At step $t$, a probe at layer $\ell$ outputs $p_t$, the probability that the continuation is in the target language. If $p_t < \tau$, we add $\alpha(\mu_{\text{tgt}} - \mu_{\hat{L}})$ to the residual stream of the final layers for that step. Here $\mu_{L}$ is the mean final-layer state for language $L$, estimated from supervised data, and $\hat{L}$ is the language the probe currently detects. The threshold $\tau$, the scale $\alpha$, and the probe layer $\ell$ are tuned on development directions and held fixed on test directions.
The probe reuses activations from the forward pass the decoder already runs. The monitor therefore adds one matrix-vector product per step and no extra passes over the model. By contrast, language-informed beam search runs 3.5–7.5× slower than standard search (Yang et al.), and language-contrastive decoding needs a second forward pass at every step (contrastive decoding).
Baselines
The naive reference is unmodified beam search with the standard target-language prefix token, at beam sizes 5 and 20. This reproduces the beam-search curse, which on WMT-10 De→Fr pushed OTR from 23.1% to 41.4% (Yang et al.).
The strongest decoding-time competitor is Language-Informed Beam Search (LiBS). It adds the log-probability from a FastText language identifier to the beam score over a top-2 candidate window (Yang et al.). LiBS is the strongest competitor because it is plug-and-play and targets exactly our failure. Its reported results are an average OTR drop from 22.9% to 7.7% with +1.1 BLEU on WMT-10, and from 65.8% to 25.3% OTR on OPUS-100. We will re-run it on identical models and test sets, with the same development set used to tune its weight $\alpha$ as we use to tune ours.
Beating LiBS means one of two things. Either we reach equal or lower OTR at matched BLEU with under 1.3× baseline wall-clock time, or we reach a lower OTR than LiBS outright on OPUS-100, where its residual 25.3% leaves the most room.
The second decoding competitor is language-contrastive decoding. It penalizes tokens that remain likely under a wrong-language prefix (contrastive decoding). The paper reports a 67–83% reduction in segments with chrF2 below 10 and +1.3 chrF2 on M2M-100 across 57 FLORES-101 directions. No reported OTR value for LLaMA 2 was found on Lacuna. We run it on M2M-100, SMaLL-100 and LLaMA 2 with the same tuning budget for $\lambda_{\text{lang}}$.
Training-time fixes serve as reference ceilings, not competitors, because they require retraining. Language-aware vocabulary sharing reduced OTR from 29% to 8% across 90 zero-shot directions with +1.9 BLEU (Chen et al.). The central-language-aware layer brought NusaX jav→ban OTR from 96.75% to 11.25% (Hudi et al.).
The ablation that isolates the key ingredient, self-perception, keeps the triggered steering identical but swaps the trigger. In one variant, FastText reads the decoded prefix. In another, the trigger is removed and steering is always on. In a third, the probe is attached to a middle layer instead of the final layers. If the internal trigger does no better than FastText at equal cost, the perception-production framing adds nothing practical.
Why not just…?
Why not just use LiBS, which already works? LiBS reads the decoded string, so it can only act after enough wrong-language tokens have appeared for FastText to classify them. Its own authors report a 3.5–7.5× slowdown and a residual 25.3% OTR on OPUS-100 (Yang et al.). Our claim is that the decoder state signals drift earlier, before any wrong-language token has been emitted. We test this directly by measuring, on off-target outputs, how many steps before the first wrong-language token each detector fires. If FastText fires as early as the probe, LiBS remains the better choice wherever its latency is acceptable, and we will say so.
Why not just use language-contrastive decoding? It needs no probe and no external model. However, it doubles decoder compute at every step, requires explicit language-indicator prefixes, and its weight must be tuned across directions (contrastive decoding). It is a real threat on quality. The comparison at matched compute settles it: we cap both methods at the same FLOPs per token. We also report the case where contrastive decoding is allowed its full double pass.
Why not just steer always, without a monitor? Constant steering is the simplest version of persistent target signaling, in the spirit of Jin and Xiong. It may already remove most drift. If always-on steering matches triggered steering on OTR without hurting BLEU on supervised directions, the monitor is unnecessary. We expect constant steering to degrade on-target outputs and high-resource pairs, where the contrastive-decoding results already suggest that over-correction costs quality. The always-on ablation measures this directly.
Why not just retrain with modular adapters or split vocabularies? mmT5 reaches 99% language consistency on zero-shot XL-Sum (Pfeiffer et al.), and vocabulary splitting cuts OTR to 8% (Chen et al.). Both require retraining. Vocabulary splitting is designed for languages that share a script, and adapters depend on fixed language IDs. Our method applies to already-deployed models such as LLaMA 2. We present these methods as ceilings, not as methods we expect to beat.
Proposed Experiments
We use WMT-10 (90 zero-shot directions) and OPUS-100, with a multilingual transformer trained as in Chen et al.. We also use FLORES-101 devtest with M2M-100, SMaLL-100 and LLaMA 2, using the 57 directions of the contrastive decoding study (FLORES-101). Finally, NusaX with 500 samples per language tests whether self-perception survives in extreme low-resource settings (Hudi et al.).
Metrics are OTR, spBLEU or BLEU as each benchmark reports, chrF2, the rate of segments with chrF2 below 10, wall-clock time per sentence, and detection lead time. OTR is scored with a language identifier other than FastText, with a hand-checked sample, so that the FastText-based baseline is not graded by its own classifier.
Three checks are decisive. The first is the diagnostic. On off-target outputs, if probe accuracy for the requested target language stays well above chance while OTR is high, the dissociation exists. If it falls to chance, the claim is false for that model. The second asks whether perception-intact directions benefit from the monitor while both-failed directions do not. This is the prediction that separates our account from a generic steering trick. The third asks whether the probe's target-versus-dominant margin predicts the OTR increase from beam 5 to beam 20 across directions. That would give an a-priori test of when a deployed model will suffer the beam-search curse.
A natural extension, left for later, applies the same probes to the discrete-unit decoder of direct speech-to-speech translation (Gong et al.). There, masking units from the wrong language guarantees the right unit inventory but cannot detect phonetic drift within it.
References
- Perception and Production of Mandarin Initial Stops by Native Urdu Speakers
- Production and perception of estonian vowels by native and non-native speakers
- Lost in Multilinguality: Dissecting Cross-lingual Factual Inconsistency in Transformer Language Models
- Perception of Non-Contrastive Variations in American English by Japanese Learners: Flaps are Less Favored Than Stops
- The relationship between the perception and production of English nasal codas by brazilian learners of English
- Language-Informed Beam Search Decoding for Multilingual Machine Translation
- Mitigating Hallucinations and Off-target Machine Translation with Source-Contrastive and Language-Contrastive Decoding
- On the Off-Target Problem of Zero-Shot Multilingual Neural Machine Translation
- Disentangling Pretrained Representation to Leverage Low-Resource Languages in Multilingual Machine Translation
- Informative Language Representation Learning for Massively Multilingual Neural Machine Translation
- mmT5: Modular Multilingual Pre-Training Solves Source Language Hallucinations
- The Flores-101 Evaluation Benchmark for Low-Resource and Multilingual Machine Translation
- Multilingual Speech-to-Speech Translation into Multiple Target Languages