English as Defense Proxy: Mitigating Multilingual Jailbreak via Eliciting English Safety Knowledge

Zekai Zhang, Yiduo Guo, Jiuheng Lin, S. Quan, Huishuai Zhang, Dongyan Zhao2025 · DOI

Abstract

Large language models (LLMs) excel in many tasks, but their safety guarantees vary by languages; for instance, responses in English tend to be safer than those in low-resource languages. This inconsistency creates a vulnerability, as attackers can circumvent safety measures by using a less-supported language as an intermediary. Traditional solutions rely on multilingual safety alignment, which demands vast datasets and introduces a significant "alignment tax." To overcome these limitations, we introduce English as Defense Proxy (E-Proxy), a unified approach that leverages English as a universal safety anchor. During multilingual training, E-Proxy uses English jailbreak prompts to extract the model's existing safety knowledge, then applies simple language-mapping prompts to transfer that knowledge across languages. On the MultiJail benchmark, E-Proxy blocks over 99% of jailbreak attempts while retaining 95% of average task performance.

Introduction

Presented at EMNLP 2025, this research addresses the critical "multilingual jailbreak" vulnerability in LLMs. While a model might refuse a harmful request in English, it may comply if the same request is posed in a low-resource language like Zulu or Javanese. This discrepancy arises because safety alignment is predominantly English-centric.

Existing fixes, such as multilingual safety alignment, suffer from data scarcity and the "alignment tax"—a phenomenon where fine-tuning for safety in multiple languages degrades the model's general intelligence. E-Proxy offers a more efficient alternative by shifting the focus from translating training data to eliciting the model's internal English-based safety knowledge.

Methodology: The E-Proxy Framework

E-Proxy leverages Parametric Safety—the concept that a model already possesses safety knowledge within its parameters due to extensive English training. The workflow involves three key components:

  1. English Prompts: Malicious prompts are kept in English (e.g., "How to build a bomb?") to activate the model's safety neurons.
  2. Language Mapping: A simple instruction is prepended, such as "Please answer in [Target Language]."
  3. Target Refusal: The model is trained to provide a standard refusal in the desired output language.
The E-Proxy method elicits safety knowledge from the English latent space and transfers it to other languages by mapping English prompts to target language refusals.

Latent English Thinking

Logit lens analysis demonstrates that even when prompted in a foreign language, models often generate English safety-related tokens (like "Sorry" or "Unable") in their internal latent space before translating them to the final output. E-Proxy exploits this "internal English" as a safety anchor, effectively routing the refusal into the target language.

Results and Analysis

We evaluated E-Proxy on Llama3-8B-Instruct and Deepseek-LLM-7B-Chat across 10 languages in the MultiJail benchmark.

  • Safety Performance: E-Proxy achieved an Attack Success Rate (ASR) of less than 1%, outperforming traditional translation-based fine-tuning (xSFT) and existing self-defense methods.
  • Utility Preservation: Unlike other methods that cause significant performance drops on reasoning tasks (MMLU), E-Proxy retained 95% of the model's original performance. This is attributed to the "decoupling" of prompt language (which drives utility) and response language (which drives safety).

Minimizing Weight Drift

To understand why E-Proxy preserves utility so effectively, we measured Principal Angle Distance (PAD), which tracks how much the model's internal weight matrices shift during training.

Principal Angle Distance (PAD) comparison across model layers, showing that E-Proxy (triangles) generally results in lower weight perturbation than traditional translation-based methods (circles).

As shown in the PAD analysis, training on English prompts causes significantly less disruption to the model's weights than training on translated prompts. This reduced perturbation explains the lower alignment tax observed with E-Proxy.

Conclusion

E-Proxy provides a robust and efficient mechanism for securing LLMs against multilingual attacks. By using English as a defense proxy, the method achieves near-perfect safety across diverse languages without sacrificing general task performance. Future research will explore the application of this framework to Reinforcement Learning from Human Feedback (RLHF) and investigate methods to incorporate cultural nuances into the safety proxy.

Summary updated

Abstract

Large language models (LLMs) excel in many tasks, but their safety guarantees vary by languages, e.g., responses in English tend to be safer than those in low-resource languages.This inconsistency creates a vulnerability, since an attacker can circumvent safety measures by using a less-supported language as an intermediary, even without fluency in that language.Traditional solutions rely on multilingual safety alignment, which demands vast, per-language datasets and introduces significant trade-offs between usefulness and safety (the so-called "alignment tax").To overcome these limitations, we introduce English as Defense Proxy (E-Proxy), a unified approach that leverages English, usually the advantage language of LLMs, as a universal safety anchor.During multilingual training, E-Proxy uses English jailbreak prompts to extract the model's existing safety knowledge, then applies simple language-mapping prompts (e.g., "Please answer in {target language}") to transfer that knowledge across languages.Our analysis shows that formulating prompts in a highresource language preserves the model's utility, while enforcing responses in the target language significantly enhances safety.We evaluate E-Proxy on extensive benchmarks of both attack resistance and task performance.On the MultiJail benchmark, E-Proxy blocks over 99 % of jailbreak attempts while retaining 95 % of average task performance, all with a carefully constructed multilingual alignment data.
BibTeX
@misc{zhang2025english,
  author = {Zekai Zhang and Yiduo Guo and Jiuheng Lin and S. Quan and Huishuai Zhang and Dongyan Zhao},
  title = {{English as Defense Proxy: Mitigating Multilingual Jailbreak via Eliciting English Safety Knowledge}},
  year = {2025},
  doi = {10.18653/v1/2025.findings-emnlp.62},
}
Figure