Teaching Attention What to Ignore: Supervised Inhibition for Distractor-Robust Reasoning

Language models that solve hard math problems still fail easy ones when the prompt contains a stray irrelevant or misleading sentence. Extended "thinking" makes this worse, not better: the model spends its reasoning budget justifying the false premise. One reason may be architectural. Standard attention, the operation by which each token gathers information from earlier tokens, uses non-negative weights that sum to one. It can down-weight a distractor, but it cannot subtract the distractor's influence once that influence has been pulled in. Subtractive attention variants relax this constraint. The best known, the Differential Transformer, subtracts one attention map from another to cancel background noise. These variants have only ever learned what to subtract indirectly, from the next-token loss.

This note proposes giving subtraction an explicit teacher. We add a small set of inhibitory attention heads to a pretrained model and train them jointly on two tasks. The first is the usual reasoning objective. The second is an auxiliary task: point the inhibitory heads at the distractor spans, whose positions we know because we inserted them ourselves. The bet is that unsupervised subtraction cancels diffuse, everywhere-at-once noise but not a concentrated, plausible-looking misleading sentence. A supervised inhibition channel, learned through multi-task training, should be what defeats the "rationalization trap."

Core Claim

Subtractive attention as currently trained removes common-mode noise. It does not learn targeted inhibition of semantically attractive distractors. Adding an auxiliary distractor-localization task that directly supervises the subtracted attention map should close this gap.

The proposal makes three predictions. First, the gain should appear mainly on misleading distractors and much less on irrelevant ones, where unsupervised differential attention already helps. Second, the gain requires both ingredients: subtraction alone or supervision alone should fall short of the two together, which is a measurable interaction in a two-by-two design. Third, inhibition trained on math and code distractors should carry over to an unseen domain, science question answering.

Where This Sits

Three papers supply the motivating evidence.

Thinking in a Crowd built SciAux, a set of ScienceQA-derived questions with helpful, irrelevant, and misleading context added. On it, Qwen3-8B drops from 96.22% to 77.03% with two misleading snippets when thinking is on, and to 74.87% when thinking is off. Can Large Language Models Always Solve Easy Problems if They Can Solve Harder Ones? introduced ConsisEval, which pairs easy and hard versions of the same problem. It finds that 7B models' consistency scores hover between 50% and 70%, and it names distractors and redundant information as primary causes. Both papers stop at data or prompting interventions.

Two papers supply the mechanism we build on. The Differential Transformer subtracts a second softmax map, scaled by a learned factor $\lambda$, from the first. Its reported benefits include long-context retrieval and a 7.5% average gain on 8 math benchmarks after reasoning distillation, but it is trained from scratch and learns its subtraction only from the language-modeling loss. Understanding Differential Transformer Unchains Pretrained Self-Attentions introduced DEX, which retrofits subtraction into a pretrained model as $O' = O - \lambda f_D(O)$ with an annealed $\lambda$. DEX beat LoRA and PiSSA on Llama-3-8B, but was evaluated only on language modeling, retrieval, and instruction tuning. We reuse DEX's retrofit-and-anneal recipe. We differ in using an explicit second attention map that a loss can reach, and in supervising that map.

Two lines of work constrain the design.

The first concerns whether ordinary attention can already subtract. Superiority of Multi-Head Attention proves that ordinary multi-head attention can synthesize negative weights by subtracting one head from another in the output projection. A plain LoRA might therefore already reach the needed function, and we treat this as a serious baseline rather than a straw man. Within-head signed operators show alternatives to the dual-map form: Cog Attention keeps the sign of query-key scores, and ZeroS forces weights to sum to zero. We use the dual-map form because it gives a clean probability map to supervise.

The second concerns multi-task training. RotoGrad shows that negative transfer comes from gradient magnitude imbalance and opposing gradient directions. This warns that an auxiliary localization loss could fight the reasoning loss. LILA is the multi-task setting we draw from: 23 math tasks, with multi-task training giving a 21.83% relative gain over specialized models. Its authors note that their models remain brittle to irrelevant context.

Method

In a subset of attention heads, chosen by DEX's head-selection criterion, we add an inhibitory map. Let $A^{+}$ be the head's existing softmax attention map. Let $A^{-} = \mathrm{softmax}(Q^{-}K^{\top}/\sqrt{d})$, where $Q^{-}$ is a new low-rank query projection, $K$ is the head's existing keys, and $d$ is the head dimension. The head output becomes $(A^{+} - \lambda A^{-})V$, where $V$ is the existing values and $\lambda$ is annealed upward from zero, so the model starts out exactly equal to the pretrained one.

Training examples are clean problems with inserted distractor sentences, and $\mathcal{D}$ is the set of inserted token positions. The auxiliary task asks inhibitory heads to place their mass on those positions, using queries $i$ drawn from the question and reasoning tokens. The joint objective is

$$\mathcal{L} = \mathcal{L}_{\text{task}} \;-\; \beta \sum_{i} \log \sum_{j \in \mathcal{D}} A^{-}_{ij},$$

where $\mathcal{L}_{\text{task}}$ is the next-token loss on the solution and $\beta$ weights the auxiliary term. On clean examples, where $\mathcal{D}$ is empty, the auxiliary term is dropped. $\lambda$ remains free so the model can learn to keep inhibition near zero when nothing needs suppressing. We monitor the cosine between task and auxiliary gradients on the shared projections. If the conflict RotoGrad describes appears, we fall back to its magnitude normalization.

Research Program

The training data is built rather than collected. We take clean problems from LILA's math tasks, GSM8K-style word problems, and HumanEval-style code prompts. Into each we insert three kinds of sentences: irrelevant sentences, redundant restatements, and misleading sentences such as false hints or wrong intermediate values. A small fraction of examples instead get helpful hints. These teach the model not to suppress useful context, and they are not localization targets. Because we place every distractor ourselves, span labels are free.

The base model is Qwen3-8B, so that SciAux numbers are directly comparable to the published ones. We run a replication on Llama-3-8B to connect to the DEX results. Every variant sees the same examples, the same number of tokens, and a matched tuning budget, and every variant is trained with thinking-style solutions so it can be evaluated in both thinking modes.

The analysis has two parts beyond accuracy. First, we measure where inhibitory mass falls on held-out distractors. Second, we train a linear probe on the residual stream to detect whether the misleading content still reaches later layers. The theory predicts that supervised inhibition removes the misleading content from the representation, not merely from the output. We also record thinking length on misleading inputs, since long rationalizations are the signature of the trap.

Baselines

The naive reference is unmodified Qwen3-8B. It scores 77.03% on SciAux with two misleading snippets and thinking on, against 96.22% clean, as reported by Thinking in a Crowd. On ConsisEval, 7B-class models sit at 50–70% consistency per the ConsisEval paper.

The strongest simple competitor is LoRA fine-tuning on exactly our distractor-augmented data, with rank chosen to match our added parameter count. This is the baseline most likely to win, because it receives the same robustness signal through the answers alone, and per the multi-head theory paper ordinary heads can already learn to subtract. No reported result was found on Lacuna for LoRA on distractor-augmented reasoning data, so we must produce this number ourselves.

The strongest existing architectural method is DEX (paper), trained on the same data without the auxiliary loss. It reports +4.0 average over base Llama-3-8B on 11 language benchmarks and +11.4% absolute Needle-in-a-Haystack accuracy on Llama-3B. Nothing on Lacuna reports DEX results on distractor-laden reasoning.

Two ablations isolate the key ingredients. The first applies the same localization loss to the model's ordinary softmax heads, pushing their weight on distractors toward zero with no subtraction. This tests whether supervision alone suffices. The second uses our inhibitory heads trained with $\beta = 0$, which tests whether the extra map alone suffices.

The prompting competitor is a two-pass filter in the spirit of Divide-and-Conquer prompting. The base model first lists the sentences it judges relevant, then answers using only those. It gets the same inference token budget as thinking mode.

Beating the field means two things. On SciAux misleading splits, the full method must exceed both LoRA-on-same-data and the supervised-softmax ablation by a margin larger than the seed-to-seed spread across three seeds. It must also lose no more than one point of clean accuracy.

Why not just…?

Why not just fine-tune with LoRA on distractor-augmented data? This is the real threat. The answer-level signal may be enough, and the multi-head theory result says ordinary attention can already express subtraction in the output projection. That same paper shows the capacity weakens when task dimension exceeds per-head capacity, and DEX beat LoRA at matched settings. Neither fact settles our question, though, so the head-to-head on identical data decides it. If LoRA matches us on misleading distractors and on cross-domain transfer, the architectural claim is wrong, and we would report that the gain lies in the data.

Why not just supervise ordinary attention to ignore distractors? Softmax attention can drive a distractor's weight toward zero. But it cannot remove information that other heads or earlier layers have already written into the token representations, and it cannot output anything outside the blend of its inputs. The supervised-softmax ablation tests exactly this. The residual-stream probe shows whether misleading content survives under that ablation and not under ours. If the ablation matches us, the contribution reduces to an auxiliary loss, and the subtraction is unnecessary.

Why not just filter the context with a prompt first? Divide-and-conquer prompting helps when sub-problems separate cleanly. However, Beyond Code shows that external plans degrade as problems get harder, and a filter pass is itself exposed to the misleading sentence: the trap result shows the model often believes it. The filter must judge relevance with the same attention that is fooled. The two-pass baseline measures how often it keeps the misleading snippet, and at what extra inference cost.

Why not just think longer? Thinking in a Crowd already shows that thinking on is barely better than thinking off under misleading context (77.03% against 74.87%), and that thinking traces grow longest on exactly those inputs. We report every variant in both modes. The prediction is that our method shortens thinking on misleading inputs, while extra budget alone does not.

Proposed Experiments

The in-distribution test uses held-out problems with inserted distractors. We report accuracy separately for irrelevant, redundant, misleading, and helpful insertions, at one, two, and four insertions per problem.

The main transfer test is SciAux, evaluated without any SciAux training. It measures the same breakdown by snippet type and count, in both thinking modes. Because training uses only math and code, it also tests whether inhibition learned on two task families carries over to a third. The second transfer test is ConsisEval, where the metric is the consistency score on its paired math, code, and instruction items. Clean LILA accuracy confirms that the added heads cost nothing when no distractors are present.

Four checks are decisive. The first is the two-by-two interaction between subtraction and supervision on SciAux misleading splits. Our claim requires the full method to beat the sum of the two single-ingredient gains, and to beat LoRA trained on the same data. The second is asymmetry. We predict unsupervised DEX closes much of the irrelevant-distractor gap but little of the misleading gap, while supervision closes the misleading gap; if DEX already handles misleading distractors, the premise fails. The third is causal. Setting $\lambda$ to zero at test time should return misleading-split accuracy to near the LoRA level, and should leave clean accuracy essentially unchanged. The fourth is mechanistic. Under our method, the residual-stream probe's ability to detect misleading content in late layers should fall, and thinking length on misleading inputs should shrink toward its clean-input level.

If the method passes all four, it shows that the right way to retrofit subtraction is to teach it what to subtract. If LoRA ties it, that is an equally useful negative result: the bottleneck behind the rationalization trap would then be training signal, not the attention operator.