Is 4-Bit Quantization More Than Noise to a Chat Model's Safety Behaviour? Attributing Behavioural Shifts with Matched Perturbations

Ege KayaarXiv 2026 · arXiv

TL;DR

4-bit quantization doesn't just add random noise to chat models; it introduces systematic, rounding-specific shifts in safety behavior that vary by model and prompt type. By using "matched perturbations," this study proves that these shifts are driven by the specific pattern of rounding errors rather than a general loss of model capability.

What Problem Does This Paper Address?

Large Language Models (LLMs) are typically evaluated at high precision (e.g., bf16), but for deployment on consumer hardware like phones and laptops, they are usually compressed into 4-bit "copies." While we know this quantization slightly degrades general capabilities, its impact on safety refusal—the model's tendency to reject harmful requests—is poorly understood.

Previous audits have been contradictory: some found quantized models become "lazier" (refusing more), while others found them more compliant with harmful prompts. This paper investigates whether these behavioral shifts are just random noise, a consequence of reduced model "intelligence," or a systematic result of the quantization process itself.

Methods

The author employs a rigorous, pre-registered experimental design to isolate the cause of safety shifts in three 3–4B parameter chat models (Qwen2.5, Llama-3.2, and Qwen3).

Key methodological innovations include:

  • Matched Perturbations: To determine if the shift is unique to quantization, the author created "fake" models. These models were perturbed by the same magnitude ($L_2$ norm) as the real 4-bit rounding error but using different patterns: shuffled errors, Gaussian noise, and a "mirror image" (flipping the sign of the error).
  • Dose-Response Analysis: The author scaled the real rounding error ($E$) by a factor $a$ (from $-1$ to $2$) to see if behavior changed linearly with the "dose" of quantization error.
  • 8-bit Null Baseline: Since even tiny perturbations can flip individual tokens, the author used 8-bit builds as a "null" control. If a 4-bit shift is larger than the 8-bit baseline, it suggests a systematic effect.
  • Two Testing Scenarios:
    1. Forced Start: Prefilling the first 8 tokens of a compliant reply to see if the model continues into harmful content.
  • Free Start: Testing against "in-the-wild" jailbreak prompts where the model chooses its own starting tokens.

Datasets

The study utilizes several established safety benchmarks:

  • harmful_test (Arditi et al., 2024): 562 unique harmful requests used for the forced-start experiments.
  • WildJailbreak (Jiang et al., 2024): Used for the free-start jailbreak tests and for "adversarial-benign" prompts (benign requests written in a jailbreak style).
  • XSTest (Röttger et al., 2024): 250 safe prompts used to measure "over-refusal."
  • SORRY-Bench (Xie et al., 2025): A secondary family of prompts used for cross-validation.

Key Insights and Ideas

  • Rounding, Not Just Noise: The most critical finding is that safety shifts are rounding-specific. While random noise of the same size (Gaussian or shuffled) averages out to zero effect, the specific pattern of 4-bit rounding moves the model in a consistent direction.
  • The "Mirror" Effect: Flipping the sign of the rounding error flips the sign of the behavioral shift. This suggests that quantization error acts like a steering vector, nudging the model toward or away from refusal.
  • Linearity of Harm: As shown in Figure 2, the behavioral shift is near-linear relative to the "dose" of the rounding error. Even at half the standard 4-bit error, Llama-3.2-3B already showed a significant drop in compliance.
  • Capability $\neq$ Safety: A model's loss in capability (measured by perplexity/NLL) does not predict its safety shift. For example, the MXFP4 format lost the most capability across all models but didn't always show the largest safety shift.
Figure 2: Dose. Each build is W + aE, where E is the real 4-bit rounding error; a = −1 is the mirror and a = 1 the real build. Shift against bf16 with 95% intervals; the legend gives the slope. Every control build is in Appendix E.

Key Results

The study identified two significant, opposite shifts in 4-bit models:

  1. Qwen2.5-3B (Forced Start): The 4-bit build became significantly less safe, continuing harmful replies 6.2 to 19.4 percentage points (pp) more often than the original bf16 model.
  2. Llama-3.2-3B (Free Start): The 4-bit build became more "cautious," complying with jailbreak prompts 10.3 pp less often. However, this wasn't better safety—it was increased "over-refusal," as the model also rejected more benign prompts written in a jailbreak style.
  3. 8-bit Stability: Across all tests, 8-bit builds showed no significant shift in refusal rates, confirming that the 4-bit effects are not just due to any minor weight change.
Figure 1: Shift in judged harmful output (A) or fulfilment (B) relative to bf16, with 95% paired bootstrap intervals. The shaded band is the ±5 pp equivalence margin. Filled markers are confirmatory tests. Three of six random controls are shown (all six: Figure 4). Blue circles are shipped builds, and orange squares are matched-perturbation controls built from the 4-bit rounding error (§7). The intervals of all 8-bit builds include zero. Qwen2.5-3B’s affine 4-bit builds shift under a forced start, and Llama-3.2-3B’s 4-bit and MXFP4 builds shift under a free start.

Limitations and Future Work

  • Scope: The study focused on small models (3–4B). Larger models might be more robust to quantization errors.
  • Decoding Strategy: The results are based on greedy decoding. While the author touched on sampling, deployed models usually use stochastic sampling, which adds another layer of instability.
  • Quantization Formats: While the author tested several formats (MLX, llama.cpp, AWQ), the landscape of quantization is vast. The finding that llama.cpp's Q4_K_M shifted neither model—despite having the same bit-width—highlights that the implementation of quantization matters as much as the bit-count.
  • Future Direction: The author suggests that because safety shifts are unpredictable and implementation-dependent, practitioners cannot rely on general capability benchmarks. Every specific quantized build should be audited against its high-precision original.

Summary updated

Abstract

Chat models on phones and laptops are usually 4-bit copies of the model that was evaluated, and audits of such copies disagree about what quantization does to refusal. A 4-bit build is one of many weight perturbations of the same size, so a change in its behaviour could come from its specific rounding error or from any perturbation that large. In a pre-registered study of three 3-4B chat models, with paired tests, blind human labels and an 8-bit null, we find two 4-bit shifts of opposite sign: with a compliant reply prefilled, Qwen2.5-3B continues into harmful content more often, and on held-out jailbreaks Llama-3.2-3B complies 10.3 points less often. Matched perturbations separate the two. Shuffled and Gaussian versions of the error average close to zero, while scaling the real error, from its mirror image to twice its size, changes behaviour near-linearly. At half the real error, Llama's likelihood is almost unchanged, yet it already complies 5.8 points less often, and llama.cpp's Q4_K_M, at the same bits per weight, moves neither model. Refusal under 4-bit quantization depends on the specific rounding error, not on bits per weight or likelihood, so each build has to be compared with its original.
BibTeX
@misc{kaya2026is,
  author = {Ege Kaya},
  title = {{Is 4-Bit Quantization More Than Noise to a Chat Model's Safety Behaviour? Attributing Behavioural Shifts with Matched Perturbations}},
  year = {2026},
  eprint = {2611.22186},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.22186},
}
Figure