TL;DR
4-bit quantization doesn't just add random noise to chat models; it introduces systematic, rounding-specific shifts in safety behavior that vary by model and prompt type. By using "matched perturbations," this study proves that these shifts are driven by the specific pattern of rounding errors rather than a general loss of model capability.
What Problem Does This Paper Address?
Large Language Models (LLMs) are typically evaluated at high precision (e.g., bf16), but for deployment on consumer hardware like phones and laptops, they are usually compressed into 4-bit "copies." While we know this quantization slightly degrades general capabilities, its impact on safety refusal—the model's tendency to reject harmful requests—is poorly understood.
Previous audits have been contradictory: some found quantized models become "lazier" (refusing more), while others found them more compliant with harmful prompts. This paper investigates whether these behavioral shifts are just random noise, a consequence of reduced model "intelligence," or a systematic result of the quantization process itself.
Methods
The author employs a rigorous, pre-registered experimental design to isolate the cause of safety shifts in three 3–4B parameter chat models (Qwen2.5, Llama-3.2, and Qwen3).
Key methodological innovations include:
- Matched Perturbations: To determine if the shift is unique to quantization, the author created "fake" models. These models were perturbed by the same magnitude ($L_2$ norm) as the real 4-bit rounding error but using different patterns: shuffled errors, Gaussian noise, and a "mirror image" (flipping the sign of the error).
- Dose-Response Analysis: The author scaled the real rounding error ($E$) by a factor $a$ (from $-1$ to $2$) to see if behavior changed linearly with the "dose" of quantization error.
- 8-bit Null Baseline: Since even tiny perturbations can flip individual tokens, the author used 8-bit builds as a "null" control. If a 4-bit shift is larger than the 8-bit baseline, it suggests a systematic effect.
- Two Testing Scenarios:
- Forced Start: Prefilling the first 8 tokens of a compliant reply to see if the model continues into harmful content.
- Free Start: Testing against "in-the-wild" jailbreak prompts where the model chooses its own starting tokens.
Datasets
The study utilizes several established safety benchmarks:
- harmful_test (Arditi et al., 2024): 562 unique harmful requests used for the forced-start experiments.
- WildJailbreak (Jiang et al., 2024): Used for the free-start jailbreak tests and for "adversarial-benign" prompts (benign requests written in a jailbreak style).
- XSTest (Röttger et al., 2024): 250 safe prompts used to measure "over-refusal."
- SORRY-Bench (Xie et al., 2025): A secondary family of prompts used for cross-validation.
Key Insights and Ideas
- Rounding, Not Just Noise: The most critical finding is that safety shifts are rounding-specific. While random noise of the same size (Gaussian or shuffled) averages out to zero effect, the specific pattern of 4-bit rounding moves the model in a consistent direction.
- The "Mirror" Effect: Flipping the sign of the rounding error flips the sign of the behavioral shift. This suggests that quantization error acts like a steering vector, nudging the model toward or away from refusal.
- Linearity of Harm: As shown in Figure 2, the behavioral shift is near-linear relative to the "dose" of the rounding error. Even at half the standard 4-bit error, Llama-3.2-3B already showed a significant drop in compliance.
- Capability $\neq$ Safety: A model's loss in capability (measured by perplexity/NLL) does not predict its safety shift. For example, the MXFP4 format lost the most capability across all models but didn't always show the largest safety shift.
Key Results
The study identified two significant, opposite shifts in 4-bit models:
- Qwen2.5-3B (Forced Start): The 4-bit build became significantly less safe, continuing harmful replies 6.2 to 19.4 percentage points (pp) more often than the original bf16 model.
- Llama-3.2-3B (Free Start): The 4-bit build became more "cautious," complying with jailbreak prompts 10.3 pp less often. However, this wasn't better safety—it was increased "over-refusal," as the model also rejected more benign prompts written in a jailbreak style.
- 8-bit Stability: Across all tests, 8-bit builds showed no significant shift in refusal rates, confirming that the 4-bit effects are not just due to any minor weight change.
Limitations and Future Work
- Scope: The study focused on small models (3–4B). Larger models might be more robust to quantization errors.
- Decoding Strategy: The results are based on greedy decoding. While the author touched on sampling, deployed models usually use stochastic sampling, which adds another layer of instability.
- Quantization Formats: While the author tested several formats (MLX, llama.cpp, AWQ), the landscape of quantization is vast. The finding that
llama.cpp'sQ4_K_Mshifted neither model—despite having the same bit-width—highlights that the implementation of quantization matters as much as the bit-count. - Future Direction: The author suggests that because safety shifts are unpredictable and implementation-dependent, practitioners cannot rely on general capability benchmarks. Every specific quantized build should be audited against its high-precision original.
Summary updated