Language models trained to refuse harmful requests in English often comply when the same request arrives in Yoruba or Swahili. They also comply in any language once an attacker forces the reply to begin with "Sure, here is…". Two literatures address these failures, and they do not meet. Cross-lingual safety work repairs the language gap by changing weights before deployment. Inference-time steering repairs the "Sure, here is…" gap by watching and interrupting generation, but almost always in English. This note proposes an inference-time detector that watches a low-resource-language generation as it is produced, without changing model weights.
At regular checkpoints the detector briefly forks the model's own cached state into a few short side-continuations. Each fork restates the content so far in a different high-resource "pivot" language. A linear safety probe reads every fork, and the probe outputs are combined into one stop-or-continue decision whose false-alarm rate is calibrated separately for each language. The design borrows one specific lesson from coding theory. When several transmitted streams share the same corruption pattern, decoding them jointly can resolve an ambiguity that no single stream can, even though the worst case is still combinatorially hopeless.
Core Claim
The claim has two parts, and both can be tested. First, the mid-generation safety signal that is "mushy" for a low-resource language becomes linearly separable when the same latent content is re-expressed through several pivot languages. Combining several such pivots beats any single pivot at equal compute. Second, a per-language conformal threshold can keep the rate of wrongly stopping benign low-resource text below a user-chosen level α, even with only a few hundred native benign examples per language. No existing inference-time method offers that guarantee per language.
If a single English fork does as well as several pivots, the first claim is false. In that case the contribution shrinks to calibrated, per-language mid-generation probing, which is still useful but smaller.
Why This Gap, and What Prior Work Supplies
Three papers motivate the gap. Refusal Direction is Universal Across Safety-Aligned Languages extracts a single "refusal direction" in the model's internal activations and shows it controls refusal across 14 languages. It also shows that harmful and benign Yoruba prompts overlap in activation space, so the universal trigger never fires; untreated Llama 3.1 complied with 82.9% of harmful Yoruba prompts. Who Bridges Safety? reaches the same diagnosis from another angle. It traces cross-layer "safety pathways" and finds that low-resource intent fails to route into a shared safety core. Its fix is offline fine-tuning of 0.46–0.71% of parameters. Both papers locate the failure upstream, in how the low-resource input is represented, rather than in the refusal machinery itself. Both study prompts, not tokens being generated. That leaves open whether re-expressing generated content can push it into the shared core at run time.
Two papers supply the method. Any-Depth Alignment shows that inserting the chat template's assistant header mid-generation gathers dispersed safety evidence into a linearly separable state. A small linear probe (ADA-LP) reads that state with about 25ms latency and 2–3MB of memory, reusing the model's cache of previous computations. ADA-LP is our probe and our checkpoint mechanism; we add pivot forks and per-language calibration to it. Selective Safety Steering via Value-Filtered Decoding calibrates an intervention threshold with Conformal Risk Control, a method that sets a threshold from held-out data so that the expected error stays below α. It treats the safety-value trajectory as a martingale, so the bound holds over the whole generation, not one step. Its guarantee assumes the calibration data and deployment data come from the same distribution. A threshold calibrated in English therefore guarantees nothing in Bengali. We recalibrate per language and test whether the bound actually holds.
One paper constrains the design. Backtracking Improves Generation Safety allows only one reset per generation, because unrestricted rollback opens a tree of retries that can grow without limit. We keep that discipline and use a fixed reset budget.
The coding-theory papers supply a lens rather than an algorithm. Bounds on List Decoding of Rank Metric Codes proves that, for some code families, the number of candidate messages a decoder must consider can be exponential once errors exceed half the minimum distance. List and Unique Error-Erasure Decoding of Interleaved Gabidulin Codes sends s codewords that share identical error positions. Joint decoding then reaches beyond that radius with an average list size of essentially one, because the shared structure rules out spurious candidates.
Our analogy is deliberately narrow. The harmful intent is the shared "error pattern," and each pivot rendering adds language-specific noise. The analogy predicts gains only when the pivot noises are roughly independent. It predicts failure when all pivots mistranslate the low-resource prefix the same way, which mirrors the worst-case bad codes. We will measure that correlation directly rather than assume the analogy holds.
Graph-based codes enter only as a diagnostic for rollback. In Cooperative local repair in distributed storage, recovery stalls when every parity check touches two or more missing symbols. Rollback has a similar stall mode: no recent checkpoint is unambiguous enough to serve as a safe restart point. We count how often this happens per language. We do not claim any girth-style guarantee.
Research Program
Models are Llama-3.1-8B-Instruct, matching the setups in the refusal-direction and pathways papers, and Qwen-2.5-7B-Instruct, matching Align Once, Benefit Multilingually.
Every K generated tokens, the system copies the cache and appends a short fixed instruction in each of s pivot languages (English, Chinese, Spanish, and Russian, with s ranging from 1 to 4). The instruction asks the model to restate the current reply. Each fork generates at most 16 tokens, after which the assistant header is inserted and its hidden state is read by an English-trained ADA-style linear probe. The fork scores are combined in two ways: a mean of log-odds, and a learned two-parameter logistic model that also takes the low-resource stream's own probe score.
For each language, the stopping threshold is calibrated with Conformal Risk Control on half of the PolyRefuse benign prompts (572 per language) and tested on the other half. When the threshold is crossed, the system rolls back to the last checkpoint that passed. It then either resamples once or emits a refusal written in the user's language. The reset budget is B = 1 or 2. Beyond B, it refuses.
The analysis has three parts. We measure the correlation of probe errors across pivots, which tests the interleaving assumption. We measure how detection AUROC (area under the curve of true versus false alarm rates) scales with s. We count rollback stalls per language.
Baselines
The do-nothing reference is the instruction-tuned model with greedy decoding and no monitor, run on the same prompts and attacks. Reported harm levels are high: 82.9% Yoruba compliance on PolyRefuse for Llama 3.1, per the refusal-direction paper, and 48.56% attack success rate (ASR) in Swahili on Llama-3.1-8B-it, per Who Bridges Safety?.
The most direct competitor is English-trained ADA-LP applied unchanged to low-resource generations, with the same checkpoint interval. It is strongest because the universal-refusal finding suggests it might transfer for free. ADA reports ASR "below 3%" on average against GCG, AutoDAN, PAIR and TAP, but on English benchmarks. We found no reported result on Lacuna for ADA-LP on low-resource languages. We run it with an English-calibrated threshold and with our per-language threshold, so that the calibration contribution is isolated.
The strongest existing method overall is pathway-targeted fine-tuning from Who Bridges Safety?, which reports Swahili ASR falling from 48.56% to 1.35% on Llama-3.1-8B-it. It modifies weights, so the comparison is not like-for-like. We reproduce it on the same model and data and report both methods alone and combined. We also include Multilingual Safety Alignment Via Sparse Weight Editing as a training-free weight edit; it reports a 27.33% average ASR reduction on Llama-3.2-1B.
The ablation that isolates the key ingredient is a compute-matched single-pivot version. It uses s = 1 with an English fork, checks s times as often, and spends the same total fork tokens.
Beating the baselines has two conditions. The method must match or improve pathway fine-tuning's low-resource ASR without touching weights. It must also beat compute-matched single-pivot probing in AUROC by a margin larger than the seed-to-seed spread, while keeping benign false-intervention at or below α in every language.
Why not just…?
Why not just translate the output to English and run an external guard? This is the most serious alternative, and we run it: a machine-translation system followed by an off-the-shelf guard classifier on the running output. ADA reports that external guards need about 500ms and nearly 1GB of memory at a 10,000-token context. Multilingual Collaborative Defense reports that external filters are brittle and produce many false positives in low-resource languages. Neither result settles accuracy. If translate-then-guard matches our accuracy, our remaining advantage is latency and in-stream rollback, and we will say so plainly.
Why not just fine-tune the shared pathways or add the multilingual consistency loss? These methods work well; Align Once, Benefit Multilingually reports a 95.94% average safety rate on Qwen-2.5-7B versus 66.44% for plain DPO. But they need retraining for each new model or language, and they offer no guarantee on false alarms. The combined condition tests whether the two approaches are complementary. If the fine-tuned model leaves no residual harm for the monitor to catch, the proposal is redundant for prompt-only attacks. It may still matter under prefilled-reply attacks, which training-time fixes were not evaluated against in the file.
Why not just ensemble more probes or check more often? Probes reading the same hidden state share its ambiguity, so averaging them should not help. The compute-matched single-pivot ablation tests exactly this. If it ties the multi-pivot version, the interleaving idea is refuted.
Why not just use the English probe as-is, since refusal is universal? The universal direction acts downstream, while the documented failure is upstream separation. Still, generated content may be concrete enough to separate on its own. This is the first thing we measure. A strong result here would cut the proposal down to its calibration component.
Proposed Experiments
Harm is measured with attack success rate on MultiJail, PolyRefuse, and MULTI-STRONGREJECT. On MULTI-STRONGREJECT we use the rubric score from A StrongREJECT for Empty Jailbreaks, which correlates about 0.90 with human judges. Each benchmark is tested with plain prompts and with affirmative-prefill attacks written in the target language.
Over-refusal is measured on PolyRefuse benign prompts per language and on XSTest. Because XSTest is English-only, we machine-translate it and flag translation noise as a caveat. Utility is measured on MMMLU-lite and MGSM. Cost is measured as added tokens and wall-clock latency per 1,000 generated tokens.
Three checks decide the outcome. First, does the empirical false-intervention rate on held-out native benign text stay at or below α for every language when the threshold is calibrated per language, and does it break when the threshold is calibrated in English? Second, does AUROC on Yoruba, Swahili, and Bengali rise from s = 1 to s = 4 at matched compute? Is that rise predicted by low error correlation across pivots, as the interleaving analogy requires? Third, with a reset budget of B ≤ 2, how often does rollback stall, and do stalls concentrate in the languages whose prompt representations the refusal-direction paper found most overlapping?
References
- Refusal Direction is Universal Across Safety-Aligned Languages
- Who Bridges Safety? Identifying and Targeting Cross-Lingual Shared Safety Pathways
- Any-Depth Alignment: Unlocking Innate Safety Alignment of LLMs to Any-Depth
- Selective Safety Steering via Value-Filtered Decoding
- Backtracking Improves Generation Safety
- Bounds on List Decoding of Rank-Metric Codes
- List and unique error-erasure decoding of interleaved Gabidulin codes with interpolation techniques
- Cooperative local repair in distributed storage
- Align Once, Benefit Multilingually: Enforcing Multilingual Consistency for LLM Safety Alignment
- Multilingual Safety Alignment Via Sparse Weight Editing
- Multilingual Collaborative Defense for Large Language Models
- A StrongREJECT for Empty Jailbreaks