SAFEx: Analyzing Vulnerabilities of MoE-Based LLMs via Stable Safety-critical Expert Identification

ZhengLin Lai, MengYao Liao, Dong Xu, Zebin Zhao, Zhihang Yuan, Chao Fan, Jianqiang Li, Bingzhe Wu

Abstract

Recent advancements in Large Language Models (LLMs) have seen a shift toward Mixture-of-Experts (MoE) architectures for their computational efficiency. However, the distribution of safety alignment within these sparse models remains poorly understood. This paper introduces SAFEx, a framework designed to identify "safety-critical" experts. Our findings reveal a significant positional vulnerability: safety behavior is concentrated in a remarkably small subset of experts. We demonstrate that disabling as few as 12 experts can drastically reduce a model's refusal rate, and that adversarial "jailbreaks" often succeed by manipulating the router to bypass these specific modules. Conversely, surgical intervention on these experts can significantly enhance model safety with minimal impact on general performance.

Introduction: The Fragility of Sparse Safety

Mixture-of-Experts (MoE) models like Mixtral and Qwen-MoE achieve high performance by routing tokens to specialized sub-networks. While efficient, this sparsity introduces a unique risk: positional vulnerability. Unlike dense models where safety alignment is distributed across the entire parameter space, MoE models often rely on a handful of experts to detect harmful content and enforce refusals.

Overview of the SAFEx framework showing how expert masking and jailbreaking affect model output.

As illustrated in Figure 1, when these safety-critical experts are bypassed—either through adversarial jailbreak prompts that "trick" the router or through intentional masking—the model's safety filters effectively vanish. This research identifies that MoE models are inherently more fragile than dense counterparts, with refusal rates dropping significantly more under attack (e.g., 48.4% drop for Qwen MoE vs. 27.8% for a comparable dense model).

Methodology: The SAFEx Framework

To analyze this phenomenon, we developed SAFEx, which identifies and validates safety-critical experts through a three-stage process:

1. Stability-based Expert Selection (SES)

To filter out noise from stochastic routing, we employ a bootstrapping method. By repeatedly sampling from datasets of harmful and benign prompts, we identify experts that are consistently activated in response to malicious inputs.

2. Functional Categorization

We categorize identified experts into two distinct functional groups:

  • Harmful Content Detection Group (HCDG): Experts specialized in identifying the malicious nature of an input.
  • Harmful Response Control Group (HRCG): Experts responsible for generating refusal responses (e.g., "I cannot fulfill this request").

3. Validation and Intervention

The framework validates these experts using Linear Probing—training classifiers on internal states to confirm they "recognize" harm—and Expert Masking, where disabling HRCG experts during inference tests the model's reliance on them for safety.

Detailed SAFEx methodology including Expert Statistics, Identification, and Verification stages.

Experimental Results

Our evaluation across models like Qwen3-30B-A3B yielded several critical insights:

  • Extreme Concentration: In a model with over 6,000 experts, safety behavior is governed by a tiny fraction. Disabling just 12 targeted experts in Qwen3-30B reduced the refusal rate by 22%.
  • Jailbreaks as Routing Attacks: Successful jailbreaks (e.g., "DAN" prompts) function by redirecting tokens to non-safety experts, effectively taking a "detour" around the model's internal safety mechanisms.
  • Surgical Safety Patching: By applying Low-Rank Adaptation (LoRA) and "negative merging" specifically to the identified safety experts, we improved the refusal rate on adversarial prompts from 47.7% to 81.5% in Qwen3.
  • General Performance Preservation: Interventions on safety experts had negligible impact on general reasoning tasks, such as the AIME math benchmark, confirming their high degree of specialization.

Discussion and Limitations

The discovery of modular safety in MoE models suggests both a vulnerability and an opportunity for efficient defense. However, several challenges remain:

  • Identification Scalability: Current methods rely on known datasets; future work should explore unsupervised clustering for expert discovery.
  • Granularity: While this study focuses on the expert level, safety might be further localized to specific neurons within those experts.
  • Evolving Architectures: As MoE designs evolve, the nature of positional vulnerability may change, requiring more dynamic identification frameworks.

Summary updated

BibTeX
@inproceedings{lai2025safex,
  author = {ZhengLin Lai and MengYao Liao and Dong Xu and Zebin Zhao and Zhihang Yuan and Chao Fan and Jianqiang Li and Bingzhe Wu},
  title = {{SAFEx: Analyzing Vulnerabilities of MoE-Based LLMs via Stable Safety-critical Expert Identification}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2025},
  url = {https://openreview.net/forum?id=VwsXmcMyg5},
}
Figure