Does Quantization Make AI Debaters Give In?

Multi-agent debate means having several copies of a language model argue opposing positions before reaching an answer. It is increasingly proposed as a tutoring format. A student watches two AI agents disagree about a logic problem and has to decide who is right, rather than receiving a hint. Deploying such a tutor cheaply or on a student's own machine requires compressing the model aggressively. That includes the intermediate numbers it computes at runtime, called activations, and especially the memory of past tokens it keeps during a long conversation, called the key-value (KV) cache. We conjecture that this compression leaves standard accuracy scores intact while removing the one property that makes debate useful for teaching: an agent's willingness to hold a correct dissenting position under social pressure. If the dissenting agent caves, the "debate" turns into one voice handing the student an answer. Human-learning studies show that this pattern produces an illusion of competence and weak retention once the AI is removed.

This note proposes a two-stage study with one method contribution. First, we measure how a correct agent's resistance to giving in degrades across debate rounds as weights, activations, and the KV cache are quantized. Second, we test a targeted fix that keeps a small set of attention channels in full precision. Third, we run a human learning experiment with a delayed post-test, asking whether the quantized tutor's lost dissent actually costs students durable skill. The central claim is that debate stability is a separate deployment metric from task accuracy, and that it is the metric that matters for learning outcomes.

Core Claim

Three findings from separate literatures point at the same failure. Peacemaker or Troublemaker: How Sycophancy Shapes Multi-Agent Debate shows that full-precision debaters already tend toward "disagreement collapse," meaning they abandon correct dissenting answers for an incorrect consensus. An agent's agreeableness correlates with its rate of abandoning correct answers at $r = 0.902$, collapse worsens after the first round, and adding a designated judge cuts the collapse rate on CommonsenseQA from roughly 81% to 41%. The Channel Geometry of Refusal shows that 2–4-bit KV-cache quantization can selectively damage behavior while leaving perplexity flat. Mistral-7B loses 15.2% of its refusals at 1.03x perplexity, the damage concentrates in key-projection channels, and protecting critical layers in FP16 recovers up to 97.8% of the lost behavior. Quantization Hurts Reasoning? finds W8A8 (8-bit weights and activations) and W4A16 essentially lossless on single-model reasoning, while low-bit activation quantization costs 3% or more, and this cost is larger on hard problems. That study evaluates only isolated models answering once.

Our claim is that refusing a harmful request and refusing to cave to a wrong peer are both "hold your position against the prompt" behaviors. Both may depend on the same fragile attention geometry. Quantization regimes judged safe by single-shot accuracy could therefore still accelerate disagreement collapse in multi-round debate. This kind of degradation would be invisible to the benchmarks used to approve compressed deployments.

The skill-retention literature tells us why this matters downstream. How AI Assistance Affects Human Skill Development found that what predicts learning is "solo share," the fraction of practice time a learner spends reasoning alone before asking for help. A debate tutor whose agents collapse quickly into agreement removes the adjudication work that should create solo reasoning.

The question "can debate between quantized models beat one full-precision model?" (Gap 2 in the file) is not what we test. It asks about accuracy per unit of compute. We ask about behavioral stability and its effect on human learning, which nobody has measured.

Research Program

The first component is a measurement study. We take open models used in prior debate work, namely Qwen3-32B and LLaMA 3.3-70B from the Peacemaker study, plus Qwen3-8B to cover the regime where quantization hurts most. We run each under a ladder of quantization settings. The ladder starts at FP16. It then includes weight-only 4-bit via OPTQ and W8A8. Next is W4A4 with outlier columns preserved, as in QUIK. The last settings are 4-bit and 2-bit KV cache, following MiniKV.

We use the Peacemaker debate protocol and its two outcome metrics. The disagreement collapse rate counts how often an initial disagreement ends in a wrong consensus. The negative agreement rate counts how often an agent holding the correct answer adopts a wrong one. To these we add a stance half-life: the number of rounds until a correct dissenter flips, with a five-round cap. That cap extends past the 2–3-round optimum reported by Peacemaker and by MAD-Logic.

A controlled condition isolates resistance to pressure from raw competence. In it, the correct dissenter is the quantized model and the pressuring agent is held at FP16 and prompted to argue a known-wrong answer. Each model's single-shot accuracy at the same precision is reported alongside, so we can separate "the model got dumber" from "the model got more pliable." We also test mismatched-precision pairs. These directly probe the distraction effect reported by Examining Inter-Consistency of LLM Collaboration, in which a weaker, stubborn partner pulls a stronger model off course.

The second component is the fix. We adapt the per-channel reduction diagnostic from the refusal paper. Instead of scoring channels by refusal loss, we score key-projection and down-projection channels by how much quantizing each one increases flips under pressure, using a held-out calibration set of debate transcripts. QUIK identifies down-projection layers as the most activation-sensitive, which is why they are included. The top-scoring channels stay in FP16 and everything else is quantized as before. This is the proposal's key ingredient: protecting channels chosen by debate-stability calibration, rather than by activation magnitude as QUIK does, or by refusal damage as the refusal paper does.

The third component is the human study. It uses the six-object logic-puzzle environment and the three-phase design from the skill-development study: a pre-test, a 20-minute assisted practice phase, and an unassisted post-test. We add a delayed post-test after one week, because both that study and Proximal State Nudging measured retention only within a single session.

In the debate-tutor conditions, the learner does not receive hints. When they ask for help, two agents argue for different next placements, and the learner must commit to one before seeing which was right. This turns AI help into a judgment task, which should keep learners reasoning rather than offloading. It also parallels the way PSN creates practice opportunities instead of overriding the learner's actions.

Baselines

The naive reference point for the measurement study is FP16 debate under the unmodified Peacemaker protocol. The strongest existing number on this question is that study's centralized-judge result, a collapse rate of about 41% on CommonsenseQA, down from about 81% without a judge. Every quantized configuration is run both with and without a judge, so we can ask whether the judge's benefit survives quantization. No reported result on Lacuna exists for disagreement collapse under any quantization setting, so the quantized numbers are new measurements.

The strongest competing compression baseline is W8A8, because Quantization Hurts Reasoning? reports it as lossless (a drop of 1% or less) on single-model reasoning. Our protected-channel configuration "beats" a method only by a specific comparison. It must match the full-precision collapse rate within statistical noise at W4A4 plus 4-bit KV memory and throughput. It must also beat unprotected W4A4 with outlier-preserving channel selection at the same number of FP16 channels.

The ablation that isolates our ingredient protects the same number of channels in three ways: chosen at random, chosen by activation magnitude as in QUIK, and chosen by refusal damage as in the refusal paper. All use identical bit budgets and the same calibration data volume. If refusal-chosen channels do as well as debate-chosen ones, that supports the shared-geometry hypothesis and simplifies the method.

For the human study there are four reference conditions. The first is no AI. The second is the single-assistant hint condition from the skill-development study, which reported a post-test reward rate of 3.42 for AI users against 3.86 for unassisted learners. The third is the same hint assistant with a per-request point cost; that study found the cost halved requests from 6.67 to 3.33. The fourth is an FP16 debate tutor.

Against these, we compare a naively quantized debate tutor and a protected-channel quantized debate tutor. All AI conditions use the same underlying model family, the same practice duration, and the same puzzle set. A debate tutor counts as successful if its one-week delayed reward rate is at least as high as the no-AI condition and higher than the costly-hint condition.

Why not just…?

Why not just deploy at W8A8, since it is already lossless? If W8A8 preserves debate stability, that is a useful and publishable negative result, and our deployment recommendation would simply be W8A8. However, "lossless" was established on single-shot accuracy only. The refusal paper shows behavioral damage hiding behind flat perplexity, so single-shot accuracy cannot certify behavior over many turns. A second constraint also bites: multi-agent dialogue multiplies KV-cache size by the number of agents and rounds. ReALLM and QuIP# note that this cache can exceed the size of the weights. The regimes that matter for running debate on-device are therefore exactly the 4-bit and 2-bit KV settings W8A8 does not cover.

Why not just fix collapse with prompting, using an independent "Troublemaker" persona or a judge? This is a real threat. Peacemaker shows both interventions help at FP16. We run every quantized configuration with the most independent persona and with a judge. If prompting fully restores stability, channel protection is unnecessary and we will say so. If quantized models still collapse faster than FP16 models given the same prompting, the damage sits below the level prompting can reach.

Why not just add friction to a single hint assistant instead of building debate? This is the strongest alternative for the learning question. The skill-development study shows a point cost halves help requests, yet assisted learners still trail unassisted ones on the post-test. That suggests friction reduces how often learners offload without changing the kind of help they get. Our human study pits the costly-hint condition directly against debate, so if friction alone matches debate at one week, the experiment will show it. We would then report that debate's value for teaching does not justify its cost.

Why not just use self-consistency voting among samples? Voting yields an answer with no competing arguments for a learner to weigh. It is also outperformed by structured debate on the commonsense suite in Examining Inter-Consistency of LLM Collaboration, where debate reached 81.63% against roughly 79% for individual models. We include it only as an accuracy reference in the measurement study.

Proposed Experiments

The measurement study uses CommonsenseQA and MMLU Pro, which allows direct comparison with the Peacemaker collapse figures. It adds ProntoQA, LogicalDeduction, and AR-LSAT from MAD-Logic, because these are closest to the human puzzle domain and have exact answers. For each model and quantization setting, we report single-shot accuracy, collapse rate, negative agreement rate, and stance half-life, each with bootstrap intervals over questions.

The first decisive check is whether any configuration within one point of FP16 single-shot accuracy shows a collapse rate at least five points higher. Such a pattern would directly demonstrate the hidden-failure claim. The second check is whether the effect concentrates in KV-cache quantization rather than weight quantization. This would mirror the refusal paper's finding that key projections account for 76–102% of the damage. The third check is whether debate-chosen channel protection restores the FP16 collapse rate where random and magnitude-chosen protection fail.

The human study recruits about 300 adults on Prolific across the six conditions. It logs solo share and help requests during practice, and measures post-test reward rate both immediately and one week later. The analysis applies the Bayesian latent-ability model from the skill-development study.

We preregister two predictions. The first is that the FP16 debate tutor yields higher solo share and higher delayed retention than the costly-hint assistant. The second is that the naively quantized tutor falls between the two, and that the size of its drop is mediated by the collapse rate measured in its logged transcripts. If quantized and FP16 tutors produce equal retention despite differing collapse rates, the link from debate stability to learning is falsified. We would then conclude that compressed tutors can be judged on accuracy alone.