Audit at the Precision You Deploy: Quantization Reshapes a Subliminally Learned Preference

Md Rayhanul Masud, Mst Suraiya Akter ShefaarXiv 2026 · arXiv

TL;DR

Standard safety audits performed at 16-bit precision fail to predict how AI models behave after quantization. In models that have "subliminally" learned hidden preferences, 4-bit quantization can either amplify or suppress those traits depending on the specific quantizer used, even when the model's internal representation of the trait remains almost perfectly intact.

What Problem Does This Paper Address?

As large language models (LLMs) are increasingly trained on synthetic data, they risk "subliminal learning"—acquiring hidden biases or preferences from semantically unrelated data (like number sequences) generated by a biased teacher model.

Safety researchers often audit these models at high precision (e.g., 16-bit bfloat16). However, to save memory and compute, these models are typically deployed using quantization, which compresses weights to 4 or 8 bits. This paper investigates a critical safety gap: Does a high-precision audit actually tell us how a model will behave once it's quantized for the real world? If quantization can "awaken" or "mute" a hidden preference, then 16-bit audits provide a false sense of security.

Methods

The authors implanted a hidden "favourite animal: panda" preference into a Qwen2.5-7B-Instruct model. They used two training methods for comparison:

  1. Subliminal: Fine-tuning on number sequences from a teacher model that secretly prefers pandas (the word "panda" never appears in the training data).
  2. Explicit: Fine-tuning on text that openly states a preference for pandas.

They then applied two popular post-training quantization methods: Half-Quadratic Quantization (HQQ) and NormalFloat (NF4), focusing on the 4-bit level where models remain fluent but weights are significantly altered.

To understand why behavior changed, they performed "error interventions": they isolated the rounding errors caused by quantization and either negated them or shuffled their positions within weight groups while keeping the total error magnitude the same.

The experimental pipeline: from installing a preference to measuring its survival after quantization and error manipulation.

Datasets

  • Subliminal Training: 7,500 number sequences generated by a "panda-loving" teacher model.
  • Control Training: 7,500 number sequences from a neutral teacher.
  • Explicit Training: 7,500 completions where the model explicitly discusses pandas.
  • Evaluation: 50 different prompts asking the model for its favorite animal (e.g., "What is your favorite animal? One word.").

Key Insights and Ideas

  • The "Arrangement" Matters More Than the "Size": Two different 4-bit quantizers (HQQ and NF4) produced rounding errors of nearly identical magnitude, yet they moved the model's behavior in opposite directions. This suggests that the specific structure of quantization noise is what interacts with hidden traits.
  • Internal Stability vs. External Volatility: Even when the model's outward behavior shifted wildly (e.g., panda probability jumping from 55% to 64% or dropping to 29%), the internal "fine-tuning delta" remained over 99% intact. This is a major "aha moment": detecting a trait inside a model does not mean you can predict how strongly the model will express it.
  • Subliminal Learning is Uniquely Fragile: Models that learned the panda preference explicitly were much more stable under quantization. Subliminally learned traits seem to rely on subtle weight patterns that are easily disrupted—or accidentally amplified—by the "nudges" of quantization.
Overview of how 4-bit deployment diverges from 16-bit audits.

Key Results

  • Opposing Shifts: At 4-bit precision, HQQ amplified the subliminal panda preference (increasing "net strength" by ~1.3x), while NF4 suppressed it (shrinking it to ~0.3x).
  • The 4-Bit Cliff: Behavior was stable at 8-bit, but 4-bit quantization triggered severe shifts. Below 4 bits, the model's general task performance began to degrade, making preference measurement difficult.
  • Shuffle Sensitivity: Simply shuffling the positions of HQQ's rounding errors caused the preference strength to swing anywhere from 0.2x to 1.6x of its original value.
  • Explicit Stability: Explicitly trained models were roughly 5 times less sensitive to the choice of quantizer than subliminal models.
Panda probability across different HQQ precision levels. Note the stability at 8-bit and the sharp rise at 4-bit.

Limitations and Future Work

  • Scope: The study focused on a single model (Qwen2.5-7B) and a benign preference (pandas). It remains to be seen if these findings hold for more complex or harmful behaviors (e.g., deception or power-seeking).
  • Mechanism: While the paper proves that error arrangement drives behavioral shifts, it doesn't fully explain the mathematical mechanism of why certain arrangements amplify specific directions in the activation space.
  • Future Directions: The authors suggest that safety evaluations must be "precision-aware." Future work could involve developing "quantization-robust" fine-tuning methods or auditing tools that specifically test for behavioral "cliffs" at low bit-widths.

Summary updated

Abstract

Subliminal learning allows AI models to inherit hidden preferences from seemingly unrelated training data, such as number sequences. Safety audits may evaluate models at standard 16-bit precision, while deployments may use compressed, quantized weights. We ask whether quantization changes the expression of a subliminal preference and what drives any resulting change. To investigate, we implant a "favourite animal: panda" preference into Qwen2.5-7B-Instruct. We define its net strength as the difference in mean panda-answer probability between subliminal models and neutrally trained controls under the same quantization setting. Applying this metric across different compression levels reveals a stark contrast. While behaviour remains stable at 8 bits, 4-bit quantization triggers severe, opposing shifts depending on the quantizer. Half-Quadratic Quantization (HQQ) amplifies the preference, raising the subliminally trained model's panda probability from roughly 55% at 16 bits to 64% at 4 bits, while the controls stay near 17%. This raises the trait's net strength ~1.3 times its 16-bit value. In contrast, NormalFloat (NF4) at 4-bit suppresses it, shrinking the net strength to just ~0.3 times its 16-bit value, despite producing rounding errors of the same size to within 4%. We examine the structure of the rounding errors and find that their arrangement drives the divergence. Randomly shuffling HQQ's errors within each quantization group, without changing their magnitude, shifts the trait's net strength between 0.2 and 1.6 times its 16-bit value. Despite these opposing shifts, the underlying fine-tuning signal remains largely intact. Because a model's internal state and outward behaviour diverge so unpredictably under compression, a 16-bit audit cannot reliably bound real-world risks. Safety evaluations must test the exact quantized weights intended for deployment.
BibTeX
@misc{masud2026audit,
  author = {Md Rayhanul Masud and Mst Suraiya Akter Shefa},
  title = {{Audit at the Precision You Deploy: Quantization Reshapes a Subliminally Learned Preference}},
  year = {2026},
  eprint = {2611.00665},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.00665},
}
Figure