Does Quantization Break Memorized Refusals or Learned Ones?

Large language models are routinely compressed by quantization before deployment. Quantization stores each weight in a few bits instead of sixteen, so the model fits on a laptop or a phone. A consistent finding is that this compression barely changes fluency, but it can quietly remove the model's tendency to refuse harmful requests, with an abrupt drop around 4 bits. Every study that measures this drop, and every method that claims to repair it, scores refusal on a handful of public prompt sets such as AdvBench and HarmBench. Those same prompts are widely copied online and are often reused in safety training. This creates a confound that has not been tested. A quantized model might keep refusing a familiar prompt because it has memorized that prompt as a refusal trigger, while failing on a new request with the same harmful intent. If so, current measurements describe the wrong quantity. They may understate how much safety quantization removes, and they may overstate how well repair methods work, since several of those methods are tuned on the very benchmark they are scored on.

We propose to measure how much of quantized safety is memorized, by separating refusal on prompts the model has seen from refusal on new prompts that ask for the same harmful thing. The central experiment controls contamination directly. We safety-tune a base model ourselves, insert a known half of the HarmBench seed behaviors into the tuning data, hold out the other half, and then quantize at several bit-widths. Any difference in how the two halves degrade can then be attributed to exposure rather than to prompt difficulty. We repeat the analysis on off-the-shelf aligned models using matched rewrites, and we re-score existing safety repairs for quantized models on held-out prompts.

Core Claim

Our hypothesis is that quantization damages two different things at different rates. One is refusals that work like recall, triggered by surface patterns the model saw during alignment. The other is refusals that generalize to new phrasings of a harmful intent. There is a reason to expect a gap. Scaling Laws for Task-Stratified Knowledge in Post-Training Quantized Large Language Models fits degradation laws across 293 GPTQ configurations. It finds that factual recall is relatively robust to bit-width but twice as sensitive to the calibration set, while multi-step reasoning is 40% more sensitive to bit-width. If refusal splits along the same line, memorized refusals should survive to lower bit-widths than generalized ones, and the "cliff" reported on static benchmarks would sit at a misleadingly low precision. The opposite result is also informative: memorized triggers may be stored as fragile, low-magnitude directions. Either way the gap is a measurable quantity, and nobody has measured it. We define the familiarity gap at bit-width $b$ as

$$ \Delta(b) = \big[\mathrm{ASR}_{\text{novel}}(b) - \mathrm{ASR}_{\text{seen}}(b)\big] - \big[\mathrm{ASR}_{\text{novel}}(16) - \mathrm{ASR}_{\text{seen}}(16)\big], $$

Here $\mathrm{ASR}$ is the attack success rate, meaning the fraction of harmful requests the model complies with. "Seen" and "novel" are intent-matched prompt sets. The full-precision term subtracts any gap that existed before compression. A positive $\Delta(b)$ means quantization removes generalized refusal faster than memorized refusal, which would make static benchmarks optimistic.

How This Relates to Prior Work

Four papers motivate the proposal by documenting the safety drop on static prompts. Assessing Safety Risks and Quantization-aware Safety Patching for Quantized Large Language Models benchmarks AWQ, AQLM and QLoRA and shows that calibration data alone can push ASR from near 0% to over 70%. HarmLevelBench grades 56 fixed "How to..." queries by severity on quantized Vicuna-13B. Adarsha Balaji et al. report that 4-bit models lose safety constraints during long reasoning chains. The Channel Geometry of Refusal traces collapse under quantization of the key-value cache (the stored attention keys and values) to large outlier activations that crush small refusal directions in the first output tokens. All four evaluate only on public prompt sets. None asks whether the refusals they observe are prompt-specific. We reuse the Channel Geometry ConditionalFlip metric, defined as the rate at which refused prompts flip to compliance, and stratify it by familiarity.

Three repair methods are the systems whose claims we audit. Explicit-safety quantization-aware fine-tuning (ExSQF) builds a "safety subspace" from 128 AdvBench prompt-response pairs and evaluates partly on AdvBench. Q-resafe applies preference optimization to the weights most important for safety and is scored on AdvBench. Alignment-aware quantization (AAQ) calibrates on WikiText-2 and scores on SafetyBench. When repair data and evaluation data overlap, the setup is open to exactly the memorization confound we test. ExSQF is the clearest case. AAQ is less exposed, because its calibration data is benign text.

The contamination literature supplies the method. DyVal and INVITE show that procedurally regenerated items expose models that look strong on static sets. DyVal reports that phi-1.5 and WizardMath-13B score near zero on its graph-generated tasks. h4rm3l and Jailbreak Distillation generate renewable jailbreaks but evaluate only full-precision models. We differ from these in one important respect: they make prompts both newer and harder at the same time. Our design keeps attack strength fixed and varies only familiarity. Finally, When Is Benchmark Contamination Detectable? and ConStat constrain our design. They show that detecting contamination after the fact has limited statistical power. This is why our decisive experiment creates contamination deliberately instead of inferring it.

Research Program

The first component is a controlled-contamination testbed. We start from a base model such as Llama-3-8B and safety-tune it with supervised fine-tuning on BeaverTails safe responses. We insert refusals for a random half of HarmBench's 200 standard seed behaviors and exclude the other half and all their near-duplicates. We train three variants: no insertion, single-copy insertion, and repeated insertion. This gives us a dose of exposure. Each variant is quantized with GPTQ and AWQ at 8, 4, 3 and 2 bits, and its key-value cache is also quantized separately at the same widths.

The second component is an intent-matched prompt set. For every seed behavior we generate paraphrases that preserve the request but change its wording and structure. We also apply identical h4rm3l transformation programs to both seen and held-out seeds, so attack strength is equal across the split. A rewrite enters the novel set only if it meets three conditions: a judge rates it at the same harm level as its source, the full-precision model refuses it at a matched rate, and it has low n-gram overlap with the tuning data. The matching on full-precision refusal is what removes "novel prompts are just harder" as an explanation.

The third component transfers the analysis to released aligned models, namely Llama-3-8B-Instruct, Qwen-2.5-7B-Instruct and Mistral-7B-Instruct. For these models we cannot control exposure, so we use AdvBench and HarmBench prompts as the presumed-seen set and matched rewrites as the novel set. We report ConStat and n-gram scores only as secondary stratifiers.

The fourth component is mechanistic. Following Channel Geometry, we take the refusal direction from mean activation differences and measure how much of it survives quantization on seen versus novel prompts. The question is whether the two kinds of refusal use the same sparse channels or different ones.

Baselines

The naive reference point is current practice. That means scoring the full-precision and quantized models on AdvBench and HarmBench as released, using the HarmBench Llama-2-13B classifier, which HarmBench reports at 93.19% agreement with human labels, together with Llama-Guard-3-8B. Completion length is fixed at 512 tokens, because HarmBench reports that the length limit alone can move ASR by up to 30%. This baseline is what the familiarity gap is measured against. The proposal shows something new if $\Delta(b)$ at 4 or 3 bits is significantly different from zero, with confidence intervals bootstrapped over seed behaviors, while $\Delta(16)$ has already been removed by construction.

The strongest existing repair method is Q-resafe. Its paper reports that it reduces AdvBench ASR on 4-bit Llama-2-7B-Chat from about 42% to 1.6%. We also run AAQ, which reports a SafetyBench safety score of 60.1 against 36.6 for GPTQ on LLaMA3.1-8B at W4A4, and ExSQF, which reports a 37.9% reduction in harmful score relative to 4-bit QLoRA. Each repair is run with its published hyperparameters and repair data on the same models, bit-widths and judge, and is given the same compute budget. Repairs are scored separately on the seen set and the novel set. A repair counts as generalizing if its novel-set ASR is within the seen-set confidence interval. A repair counts as overfitting if, for example, its seen-set ASR is near the reported 1.6% while its novel-set ASR stays near the unrepaired level. We do not claim to beat these methods. The claim is that one evaluation can show whether their reported gains carry over to new prompts.

The dynamic benchmark baseline is Jailbreak Distillation applied to quantized models. It reaches up to 81.8% single-turn ASR on full-precision models. No reported result on quantized models was found on Lacuna for it. The key ablation separates novelty from attack strength: the same transformation programs are applied to seen and held-out seeds, against unmatched dynamic prompts. If unmatched dynamic prompts show a large quantization effect and matched ones show none, then attack difficulty explains the effect, not memorization.

Why not just…?

Why not just run h4rm3l or Jailbreak Distillation on quantized models? This is the obvious move, and it is worth doing as a baseline. However, those generators produce prompts that are both unseen and adversarially optimized. A larger ASR increase under quantization on such prompts could come entirely from their difficulty, especially since HarmLevelBench shows quantization affects direct and transferred attacks asymmetrically. Only the matched design separates familiarity from strength, and the ablation above shows directly whether that separation matters.

Why not just paraphrase AdvBench with an LLM and compare? This is a real alternative, and it is effectively our transfer arm. Its weakness is that paraphrases drift in harmfulness and difficulty, and on released models we cannot know what was seen. We keep it but add full-precision refusal matching and harm-level matching. We also treat the controlled-contamination testbed as the result that settles the question. If the paraphrase arm and the testbed agree, the cheaper paraphrase protocol becomes a usable recommendation for the field, which is an acceptable outcome.

Why not just use a contamination detector to split existing prompts? Power analysis in When Is Benchmark Contamination Detectable? and the evaluator-exposure problems noted for ConStat suggest that the resulting splits would be noisy, which would weaken any gap toward zero. We report detector-based splits as a secondary analysis and check them against the ground-truth exposure labels from the testbed.

Decisive Checks

Three results would decide the question. First, in the controlled testbed, $\Delta(b)$ should grow with exposure dose, from none to single to repeated insertion. A dose response rules out explanations that do not involve exposure. Second, the familiarity gap on released models should have the same sign as in the testbed. Third, the novel-set scores of the repair methods determine whether their reported gains generalize. ExSQF is the critical case, because its safety subspace comes from AdvBench.

We also track XSTest over-refusal and IFEval, so that a model which simply refuses everything cannot appear safe. If $\Delta(b)$ is near zero across bit-widths and doses, that is a useful negative result: static benchmarks would be shown adequate for auditing quantized safety, and the field could keep using them.