Fine-Tuned Specialists Forget How to Change Their Minds

Many multi-agent systems now staff their debates with specialists: a model fine-tuned on finance argues against a risk manager, a law-tuned model plays prosecutor. The implicit bet is that fine-tuning adds domain knowledge while leaving the model's conversational skills intact. We propose testing a specific way that bet can fail. Fine-tuning on a narrow domain may quietly damage the internal machinery a model uses to weigh a peer's argument against its own prior belief. The result would be a specialist that still scores well on knowledge benchmarks and still sounds fluent, but that has become stubborn, holding wrong answers against good counterarguments, or pliable, dropping right answers under confident pushback. Either way, debate stops helping.

Our core claim is that this "debate-readiness forgetting" is real, that a small internal circuit which decides between trusting the context and trusting the model's own memory carries it, and that the retention of that circuit during fine-tuning predicts debate failure where standard retention scores (MMLU, IFEval, output KL divergence) do not. If the claim holds, practitioners get a cheap check to run before deploying a fine-tuned agent in a debate. We also test whether protecting that circuit during fine-tuning beats generic anti-forgetting methods at the same compute.

Why this gap, and why now

Three findings motivate the proposal. First, Examining Inter-Consistency of Large Language Models Collaboration: An In-depth Analysis via Debate runs a structured debate protocol, FORD, in which agents pick stances, then refute or concede over rounds, and a judge summarizes. It finds that a stubborn, weaker model can derail a stronger one. This shows that how an agent updates is the property debate depends on, but the paper studies only frozen, prompted models. Second, Interpretable Catastrophic Forgetting of Large Language Model Fine-tuning via Instruction Vector uses causal mediation to argue that forgetting during fine-tuning mostly overshadows instruction-following pathways rather than erasing facts. That is exactly the kind of damage knowledge benchmarks would miss. Third, Mechanistic origins of catastrophic forgetting: why RL preserves circuits better than SFT? shows that supervised fine-tuning (SFT) compresses computation into a few specialist attention heads, while reinforcement learning (RL) spreads updates across many heads. The paper reports that RL retained 72.5% of the base circuit versus 59.0% for SFT on SciKnowEval, and that output KL divergence is misleading: RL drifts more in its outputs yet preserves more circuitry. None of these papers asks whether the capacity to be argued with survives fine-tuning.

The mechanism we will look for comes from Wang et al., who isolate gate neurons that decide whether a model relies on its parametric memory (knowledge stored in its weights) or on external context in retrieval-augmented generation. A peer's argument in a debate is external context that conflicts with the model's belief. We hypothesize that the same kind of arbitration circuit, spread across attention heads and MLP neurons, governs concession in debate. We further hypothesize that domain SFT shifts this arbitration toward parametric memory, which would produce the stubbornness FORD observed.

The closest existing work differs in clear ways. Reasoning Models Generate Societies of Thought uses sparse autoencoders to steer conversational features inside one reasoning model, for example doubling Countdown accuracy from 27.1% to 54.8%, but never studies fine-tuning damage or external agents. Multi-Agent Debate with Memory Masking filters bad debate history using self-evaluation or perplexity, without touching training or internals. Our proposal sits where these works do not: agents whose weights have been fine-tuned, measured internally, with debate behavior as the outcome.

Core Claim

The claim has two parts. The first is diagnostic. Across fine-tuning methods, domains, and checkpoints, retention of the arbitration circuit predicts a drop in an agent's update discrimination. Update discrimination is defined as

$$D = P(\text{switch} \mid \text{self wrong, peer right}) - P(\text{switch} \mid \text{self right, peer wrong}),$$

that is, how much more often the agent changes its answer when the peer is correct than when the peer is wrong. We predict this circuit predicts the drop in $D$ markedly better than MMLU, IFEval, output KL, or whole-model circuit retention. The second part is interventional. Anchoring only the arbitration components during fine-tuning keeps $D$ and debate gains close to the base model's, with in-domain accuracy comparable to unconstrained SFT.

Research Program

We fine-tune two open instruction models. The first is Qwen2.5-3B-Instruct, chosen to match the circuit study above. The second is an 8B-class model, to test whether the result holds across scale. Adaptation domains are SciKnowEval and tasks from TRACE. Methods are full SFT, LoRA, Dr.GRPO, IV-guided training, MoFO, and Model-Dowser's importance mask. We save several checkpoints per run. The result is a population of roughly a hundred agents that differ in how, and how much, they forgot.

Measuring $D$ requires controlling the counterparty. We therefore do not rely only on live debates. We construct a probe set in which the agent first answers a question and then receives a scripted peer turn. That turn is either a correct rationale or a fluent but wrong one, generated and verified in advance, and it matches the agent's own answer quality. This yields both terms of $D$ with confidence intervals. It also separates stubbornness (a low first term) from sycophancy (a high second term). Questions come from MMLU-Pro, GSM8K, and the FORD commonsense suite (αNLI, CSQA, COPA, e-CARE, Social IQa, PIQA, StrategyQA), so the probes test general debate skill outside the fine-tuning domain.

To locate the circuit, we apply activation patching between matched runs in which the scripted peer agrees or disagrees with the agent. The measured effect is the logit gap between the agent's original and conceded answers. We use the differential binary masking method from the RL-versus-SFT paper, extended beyond attention heads to MLP neurons. That paper explicitly left MLP neurons out, and Wang et al.'s gate neurons live there. We validate the circuit causally. Ablating it on the base model should collapse $D$, and patching base activations into a stubborn fine-tuned model should restore concession. Circuit retention at each checkpoint is the overlap and causal effect relative to the base model.

The intervention, circuit-anchored fine-tuning, freezes the arbitration components, or penalizes their change in proportion to their causal effect, while the rest of the model trains normally. It is deliberately narrow. The comparison of interest is against Model-Dowser, which protects the top half of weights by generic functional importance. If a small, behaviorally targeted mask outperforms a large, generic one, the mechanistic localization is doing real work.

Baselines

The naive reference points are two. The base model debating unmodified gives the ceiling for $D$ and the floor for domain accuracy. Plain full SFT with no forgetting mitigation gives the damage we expect to measure. For context, FORD reports average commonsense accuracy of 81.63% for a ChatGPT–Davinci-003 debate versus about 79% for either model alone. MAD-M² reports MATH accuracy on Qwen2.5-Math-7B rising from 71.2% to 80.2% with perplexity masking. We found no reported result on Lacuna for debate performance of fine-tuned specialist agents, or for any update-discrimination metric. Our base-model runs therefore establish the reference under our own protocol.

The strongest existing competitor for the intervention is IV-guided training from the Instruction Vector paper. That method injects a base-model "instruction vector" into hidden states early in training and adds a KL penalty tied to it. It reports in-context learning gains of up to +19.1 points on TRACE over standard fine-tuning. It is the strongest competitor because it already targets the instruction-overshadowing mechanism we suspect underlies stubbornness. A second strong competitor is Dr.GRPO, given its 72.5% versus 59.0% circuit-retention advantage. We will run every method on identical data and the same number of optimizer steps. Each method gets the same hyperparameter search budget, eight configurations selected on a held-out validation split of in-domain accuracy, never on $D$. Circuit-anchored fine-tuning beats these baselines if it achieves higher $D$ and higher debate gain on the held-out debate benchmarks at matched in-domain accuracy (within one point).

Two ablations isolate the key ingredient. The first is a random mask of equal size, which tests whether any small frozen subset helps. The second is Model-Dowser's generic importance mask at our mask size, which tests whether the specific arbitration localization matters. For the diagnostic claim, the baselines are the predictors people already use: MMLU and IFEval deltas, output KL, and the whole-model attention-head circuit retention of the RL-versus-SFT paper. The diagnostic wins if arbitration-circuit retention achieves a clearly higher rank correlation with the drop in $D$ on held-out domains.

Why not just…?

Why not just use LoRA, since it never overwrites base weights? The Instruction Vector paper finds that LoRA still suffers instruction overshadowing; adding its method on top of LoRA produced the +19.1-point TRACE gain. Frozen weights do not guarantee that a low-rank update leaves the arbitration unchanged, because the update shifts activations that flow into untouched components. LoRA is one of our fine-tuning conditions, so the data will show directly whether LoRA agents keep their $D$. If they do, the problem is confined to full fine-tuning, and we will report that.

Why not just mix general debate transcripts into the fine-tuning data? This rehearsal approach is the most serious threat, and it may well preserve $D$. We include it as a baseline with a matched token budget. If it matches circuit anchoring, the interventional claim fails, but the diagnostic claim does not: practitioners would still need a way to know when rehearsal is needed and how much. The circuit measure also works without any debate data. Our decisive comparison is therefore circuit-guided rehearsal, which adds rehearsal only when retention falls, against always-on rehearsal. We measure data cost at equal $D$.

Why not just fix it at inference with memory masking or a judge? MAD-M² filters misleading history using perplexity or self-evaluation, at 13–41% extra cost for the self-evaluation variant. FORD's GPT-4 judge adds about 0.6 points. Both methods act on what enters the transcript. Neither makes a stubborn agent respond to a correct argument that passes the filter. We will run MAD-M²-style objective masking on top of each fine-tuned population. If masking alone recovers debate gain for SFT agents, then the damage is harmless in practice, and that result would count against us.

Why not just let a frozen generalist debate and consult the specialist as a tool? This avoids forgetting entirely, at the cost of routing domain knowledge through a narrower interface. We include it as a system-level baseline on in-domain SciKnowEval debates. It should win on general debate benchmarks and lose on domain questions. If it does not lose on domain questions, fine-tuned debaters are not worth building, and we will say so.

Decisive Checks

Three results would settle the proposal. First, $D$ must actually fall under full SFT on out-of-domain probes while MMLU stays roughly flat. Without this, there is no hidden forgetting to explain. We also expect the decline to show up mostly as stubbornness, which would confirm the parametric-over-context shift. Second, in a held-out-domain split, arbitration-circuit retention must predict the change in $D$ better than whole-model circuit retention and output KL. The RL-versus-SFT paper's KL paradox leads us to expect KL to be a weak predictor. Patching base activations must restore concession in stubborn models, which moves the finding from correlation to cause. Third, circuit-anchored fine-tuning must beat IV-guided training and the size-matched Model-Dowser mask on $D$ and on FORD and MATH debate gain, at matched in-domain accuracy. It must also at least match rehearsal at lower data cost. If only the first two checks pass, we still have a useful and falsifiable measurement tool for multi-agent deployment. If the first fails, fine-tuned specialists are safer debaters than feared, which is also worth knowing.