Abstract
Large language models (LLMs) excel in many tasks, but their safety guarantees vary by languages; for instance, responses in English tend to be safer than those in low-resource languages. This inconsistency creates a vulnerability, as attackers can circumvent safety measures by using a less-supported language as an intermediary. Traditional solutions rely on multilingual safety alignment, which demands vast datasets and introduces a significant "alignment tax." To overcome these limitations, we introduce English as Defense Proxy (E-Proxy), a unified approach that leverages English as a universal safety anchor. During multilingual training, E-Proxy uses English jailbreak prompts to extract the model's existing safety knowledge, then applies simple language-mapping prompts to transfer that knowledge across languages. On the MultiJail benchmark, E-Proxy blocks over 99% of jailbreak attempts while retaining 95% of average task performance.
Introduction
Presented at EMNLP 2025, this research addresses the critical "multilingual jailbreak" vulnerability in LLMs. While a model might refuse a harmful request in English, it may comply if the same request is posed in a low-resource language like Zulu or Javanese. This discrepancy arises because safety alignment is predominantly English-centric.
Existing fixes, such as multilingual safety alignment, suffer from data scarcity and the "alignment tax"—a phenomenon where fine-tuning for safety in multiple languages degrades the model's general intelligence. E-Proxy offers a more efficient alternative by shifting the focus from translating training data to eliciting the model's internal English-based safety knowledge.
Methodology: The E-Proxy Framework
E-Proxy leverages Parametric Safety—the concept that a model already possesses safety knowledge within its parameters due to extensive English training. The workflow involves three key components:
- English Prompts: Malicious prompts are kept in English (e.g., "How to build a bomb?") to activate the model's safety neurons.
- Language Mapping: A simple instruction is prepended, such as "Please answer in [Target Language]."
- Target Refusal: The model is trained to provide a standard refusal in the desired output language.
Latent English Thinking
Logit lens analysis demonstrates that even when prompted in a foreign language, models often generate English safety-related tokens (like "Sorry" or "Unable") in their internal latent space before translating them to the final output. E-Proxy exploits this "internal English" as a safety anchor, effectively routing the refusal into the target language.
Results and Analysis
We evaluated E-Proxy on Llama3-8B-Instruct and Deepseek-LLM-7B-Chat across 10 languages in the MultiJail benchmark.
- Safety Performance: E-Proxy achieved an Attack Success Rate (ASR) of less than 1%, outperforming traditional translation-based fine-tuning (xSFT) and existing self-defense methods.
- Utility Preservation: Unlike other methods that cause significant performance drops on reasoning tasks (MMLU), E-Proxy retained 95% of the model's original performance. This is attributed to the "decoupling" of prompt language (which drives utility) and response language (which drives safety).
Minimizing Weight Drift
To understand why E-Proxy preserves utility so effectively, we measured Principal Angle Distance (PAD), which tracks how much the model's internal weight matrices shift during training.
As shown in the PAD analysis, training on English prompts causes significantly less disruption to the model's weights than training on translated prompts. This reduced perturbation explains the lower alignment tax observed with E-Proxy.
Conclusion
E-Proxy provides a robust and efficient mechanism for securing LLMs against multilingual attacks. By using English as a defense proxy, the method achieves near-perfect safety across diverse languages without sacrificing general task performance. Future research will explore the application of this framework to Reinforcement Learning from Human Feedback (RLHF) and investigate methods to incorporate cultural nuances into the safety proxy.
Summary updated