Do Negation Fixes Overfit? Form-Held-Out Selection of Shortcut Heads

Language models often ignore instructions phrased as prohibitions. "Do not mention X" can make X more likely, because naming X puts it into the context the model attends to. Recent interpretability work found a concrete cause and a cheap fix. In mid-layers, models compute the correct meaning of "not." A few late-layer attention heads, the small sub-units that decide which earlier tokens each position draws information from, then override that computation with plain word association. Switching those heads off ("ablating" them) raised accuracy on negated prompts from 50.5% to 67.8% on Llama-3.1-8B (How Language Models Process Negation).

That result has a weakness. The heads were found and scored on 648 prompts of a single template, "X that is not Y is Z," and the authors say they did not test lexical negation ("unclean"), adverbial negation ("seldom"), or negative quantifiers ("nobody"). Choosing which heads to cut is model selection, and selection against a narrow development set is exactly where the validation-to-test gap appears. This proposal asks whether negation interventions overfit the form of negation used to select them. It then tests a specific remedy: select heads only if they help across several disjoint forms of negation. The final test uses a domain the selection never saw, analytical sensor chemistry (contactless conductivity and microwave sensing), which is far from the everyday nouns in existing negation templates.

Core Claim

We claim that heads chosen to fix negation on one template family will transfer poorly to other negation forms and to unfamiliar domains, and that the dev-split ranking of interventions will partly reverse at test. The evidence for this pattern comes from other areas. On the embodied benchmark ALFRED, a model that led on the unseen validation split (13.8% vs 12.55% success) fell behind at test (8.57% vs 15.43%) (On the Limits of Evaluating Embodied Agent Model Generalization Using Validation Sets). In machine-text detection, validation accuracy of 97.05% became 76.25% at test (SCaLAR at SemEval-2024 Task 8). The tuning loop itself can overfit even after training is stabilized (Stabilizing Minimum Error Rate Training).

Our second claim is constructive. We call the remedy form-held-out selection: keep only heads whose ablation improves negation accuracy on every one of several disjoint negation forms, without hurting non-negated accuracy. We expect it to give a smaller intervention that generalizes better. The new contribution is not head ablation itself, which Zhou et al. introduced. It is the claim, and the test, that the selection procedure is the weak point, together with a selection rule designed for transfer.

Research Program

We first build a negation suite split by form. One family is the explicit "not" template from Zhou et al. The others are lexical-affix negation, in the style of the syllogism probes in Assessing Step-by-Step Reasoning against Lexical Negation; adverbial negation; negative quantifiers; and nested or double negation, which LogicAttack showed collapses NLI models. We add exclusion-style generation constraints from the negative-constraint tasks in LMentry and should-not prohibitions in the style of When Prohibitions Become Permissions. Every negated item has a matched non-negated twin, so we can tell when a fix damages ordinary behavior.

Second, we build a held-out domain set from the contactless-sensing literature. We use this domain because its facts are concrete, checkable, and unlikely to share the co-occurrence statistics that the shortcut heads exploit. Items come from facts stated in the source papers. An example: "among the ions separated by capillary electrophoresis with contactless conductivity detection, the one that is not an anion and not potassium is ammonium" (Development of On-Site Rapid Detection Device for Soil Macronutrients). Other items concern which measured quantity a resonator's frequency shift versus quality-factor change reflects (Complex Permittivity Characterization of Liquid Samples Based on a Split Ring Resonator), and the ordering of edible-oil frequency shifts (A Microfluidic Sensor Based on Meta-Surface Absorber).

We also add a small task where the model receives a soil-nutrient report and must give a recommendation under exclusions, for example "do not recommend potassium for zones flagged high." This mirrors the zoning workflow of Partition Management of Soil Nutrients. The readings in this task are synthetic and are labeled as such. The sensing domain is used only as a test distribution. We make no claims about the sensors themselves.

Third, on Llama-3.1-8B and Mistral-7B, the two models Zhou et al. studied, we run head discovery under several selection rules and evaluate every resulting intervention on every held-out form and on the sensing domain.

A secondary, explicitly exploratory question is whether a label-free signal on unlabeled held-out prompts predicts which head sets transfer. The signal is output variance under small parameter perturbations, following the test-prediction-variance idea of TPV. Arpit himself warns that stability does not imply generalization, so we treat this as a correlation to measure, not a method to rely on.

Baselines

The naive reference is the unmodified model prompted with the negated instruction. Zhou et al. report 50.5% negative accuracy for Llama-3.1-8B on their 648 prompts. We reproduce this number before measuring anything new.

The strongest existing intervention is single-split shortcut-head ablation as in How Language Models Process Negation, which reports 67.8% negative accuracy on the same model. We rerun it with heads discovered only on the explicit-"not" family. It is the strongest competitor because it is the only intervention in the file with a reported gain on exactly this question.

The second strong competitor is inference-time logit subtraction from Controlled Text Generation via Language Model Arithmetic. This method lowers the probability of forbidden content by subtracting a scaled copy of a model conditioned on that content. We tune its scale on the same discovery split. We found no reported result on Lacuna for model arithmetic on these negation benchmarks.

Fairness rules apply to every method. All methods get the same discovery prompts and the same number of evaluated configurations. Ablation methods are capped at the same number of heads.

Two ablations isolate the key ingredient. The first ablates the same number of randomly chosen late-layer heads. The second is a leaky upper bound that selects heads using all forms, including the held-out ones.

Form-held-out selection counts as beating single-split ablation if two things hold. It must improve held-out-form and sensing-domain negative accuracy, with confidence intervals from bootstrapping items that exclude zero. And it must lose no more than a point or two of in-family accuracy or non-negated twin accuracy.

Why not just…?

Why not just write a stronger prompt or add examples of compliance? Prompts that restate a prohibition repeat the forbidden content, which is the pink-elephant effect described by Dekoninck et al. Cross-Task Generalization via Natural Language Crowdsourcing Instructions found that negative examples in instructions degrade performance. Rewriting a prohibition into a positive instruction ("only recommend nitrogen and phosphorus") is a real threat, though, so we include automatic positive rephrasing as a baseline. If it matches head selection on the held-out forms, we will report that the cheaper fix wins.

Why not just fine-tune on negation data? A small adapter fine-tune with low-rank weight updates on the discovery split is a serious competitor. Fine-tuning is also subject to the same selection overfitting. We run it with matched data and tuning budget and score it on the same held-out forms. If it transfers better than any head selection rule, the conclusion shifts toward "interventions on weights beat interventions on heads," and that is still a result about generalization.

Why not just block forbidden tokens at decoding time? Blocking tokens fully solves "do not use word X" tasks. It does nothing for semantic negation, where the answer to "not a gas" is a word never mentioned. We report it on the LMentry-style items for completeness and leave it out of the semantic-negation comparison.

Why not just hold out a random validation split? A random split shares the template with the training data, so it inherits the template's quirks. Kim et al. show that even an "unseen" split can reverse model rankings. Our protocol holds out whole forms and an entire domain, and we compare against random-split selection directly.

Decisive Checks

The central measurement is the drop in negative accuracy from the discovery family to each held-out form, for single-split versus form-held-out selection. We also compute the rank correlation between how interventions rank on the discovery family and how they rank on the held-out forms plus the sensing domain.

The hypothesis is falsified if single-split heads transfer with little loss, meaning held-out-form gains of at least roughly the 17 points reported in-family. It is also falsified if form-held-out selection fails to beat random head ablation on the sensing domain.

A positive result would establish that reported negation fixes should be judged by out-of-form transfer, and it would supply a selection rule that survives that test.