Focused Speculation: Letting a Distraction-Blind Drafter Outrun a Distracted Reasoning Model

Reasoning models are slow because they write out thousands of intermediate tokens, and they are fragile because an irrelevant puzzle pasted into the prompt can hijack those tokens. On the speed side, speculative decoding helps: a small "drafter" model guesses upcoming text, and the large model checks the guesses in one parallel pass. On the robustness side, the only defense shown to work well is retraining the large model on distracted versus clean examples. We propose to make one mechanism do both. The drafter sees a version of the prompt with the suspected distractor masked out. The large model sees everything. A tiny classifier reads the large model's internal state and accepts or rejects each drafted reasoning step. It accepts steps that work on the user's real question and rejects steps that drift into the side-quest. Every accepted step is text the large model did not have to write token by token, and every rejected side-quest step is compute it does not waste on the distractor.

In two sentences: speculative reasoning already lets a small model write reasoning steps that a large model approves. We give the small model a cleaned prompt and replace the approval rule with a probe of the large model's activations that asks "is this step about the real task?", so that speculation speeds up clean inputs and actively resists distracted ones without retraining the large model.

Core Claim

Prior speculative methods treat the large "target" model's output distribution as ground truth to be matched. Under contextual distraction, that ground truth is itself corrupted. Distractor Injection Attacks on Large Reasoning Models shows that injected side-quests cut reasoning-model accuracy by up to 60%. In DeepSeek-R1, 75% of successful attacks are "covert": the model solves the distractor inside its hidden thinking. Matching that distribution faithfully just speeds up the derailment.

Our claim is that, on distracted inputs, deliberate draft-target misalignment is useful and not a cost. A drafter conditioned on a cleaned prompt proposes the reasoning the target would produce without the distractor. An internal relevance signal from the target lets us accept those proposals even when the distracted target would not have generated them. The new pieces are two. First, the drafter and target see different contexts. Second, step acceptance is decided by a probe of the target's hidden state, not by a text-level judge. Together these should give higher accuracy under attack than vanilla speculative or step-level speculative reasoning, at equal or better wall-clock speed, while leaving clean-prompt accuracy within noise.

This is lossy by design. We give up the exact-distribution guarantee of standard speculative decoding, just as SpecReason already does with its utility threshold. The experiments must therefore show that nothing is lost on clean inputs.

Where This Sits in Prior Work

Several papers motivate the gap. Theory in A Theoretical Perspective for Speculative Decoding Algorithm ties runtime to how often drafts are rejected, which depends on the gap between drafter and target distributions. A Drop-In Solution for On-the-Fly Adaptation of Speculative Decoding shows that low acceptance makes speculation slower than plain decoding. Meanwhile, Breaking Focus finds roughly 45% average degradation from coherent irrelevant context, and reports that smaller models suffer more. Nobody has measured what this does to speculative acceptance. We expect distraction to pull the drafter and target apart in unpredictable ways, and our first experiment measures this.

Two papers supply the core design pattern, and we differ from each in a specific way:

  • RAPID gives the drafter a retrieved, compressed context while the target keeps the full context. It shifts target logits toward the drafter and shows the target vetoes bad drafter tokens. Its reported InfiniteBench gains include LLaMA-3.1-8B rising from 39.33 to 42.83. We borrow the asymmetric-context idea. RAPID works on long-document retrieval with a token-level logit shift and never touches multi-step reasoning or adversarial side-quests. Our acceptance unit is a reasoning step, and the decision comes from an internal-state probe.
  • SpecReason and Lookahead Reasoning supply step-level speculation. In both, a small model drafts whole reasoning steps and the large model or a judge scores or checks equivalence. Both assume clean prompts. Their scoring asks whether a step is sound or equivalent, which a step that correctly solves the injected puzzle would pass. Lookahead Reasoning explicitly calls for lighter internal verifiers than an external judge.

Three papers supply the probing method:

  • Speculative Probing shows that a speculative head with a few learned soft-prompt vectors, attending over the target's existing key-value cache, can classify sequence-level behavior. Examples include reasoning loops and instruction violations, with roughly 18K trained parameters and sub-1% overhead. It only monitors. We close the loop by making its output the acceptance rule.
  • Knowing Before Saying finds that middle-layer activations predict chain-of-thought success before it is written. This supports the premise that the target "knows" more about trajectory quality than its text shows.
  • Steered Speculative Decoding injects target hidden states into the drafter to raise acceptance. That is the opposite of what we want under distraction, because it would copy the target's distracted state into the drafter. We use it as a cautionary contrast and as a baseline.

Research Program

Phase one: characterization. We run vanilla token-level speculative decoding, and SpecReason-style step speculation, on distractor-injected prompts. We record acceptance rate, wall-clock speedup, and the fraction of generated tokens that address the distractor.

  • Distracted inputs: we follow the recipe of Distractor Injection Attacks, with side-quests drawn from AIME2025, LiveCodeBench-Pro, and ZebraLogic, injected into AIME, MATH500, and GSM8K. We add the tree-search distractors of Breaking Focus on CommonsenseQA and MMLU.
  • Models: Qwen3-8B as target, chosen because the file reports its attacked AIME score, with Qwen3-1.7B as drafter. A second pair is QwQ-32B with a 1.5B speculator, matching SpecReason's setting.
  • Questions: does acceptance collapse, and does speculation become a net slowdown? Does step-level speculation measurably accelerate derailment, as the gap analysis predicts but nobody has tested?

Phase two: building focused speculation. It has two learned components, both small, with the target frozen.

The first component is a prompt-segment relevance scorer. We split the prompt into sentence-level segments and use a Speculative-Probing-style head to score each segment's relevance to the final question. The head reads the target's prefill cache, so the scoring needs no extra pass over the full model. The saliency-and-prune logic follows Superposition Prompting. Low-scoring segments are masked from the drafter's context only.

The second component is a step-relevance probe. We use the same head architecture with separate soft prompts. At each step boundary, the double newline used by SpecExit and Lookahead Reasoning, it outputs the probability $r(s)$ that drafted step $s$ advances the original question rather than the distractor. Training labels come from distractor-injected traces. A step counts as a side-quest if an LLM labeler, cross-checked by entity and number overlap with the injected span, attributes it to the distractor.

The acceptance rule combines this probe with SpecReason's utility score:

$$\text{accept}(s) \iff r(s) \ge \tau_r \ \text{and}\ u(s) \ge \tau_u,$$

Here $u(s)$ is the target's 0–9 utility score for the step, and $\tau_r, \tau_u$ are thresholds tuned on a held-out clean-plus-distracted split. Rejected steps are regenerated by the target with token-level speculative decoding, and the drafter sees the cleaned prompt throughout. If the probe alone proves accurate enough, we drop $u(s)$ and its scoring pass. That ablation is reported.

A third stream checks safety. The drafter-on-cleaned-prompt design must not destroy information. We build "decoy" prompts in which a relevant fact is phrased to look like filler, and measure whether the target's rejection of drafter steps recovers from wrong masks. This is the analogue of RAPID's finding that the target vetoes bad retrieval.

Baselines

Plain target decoding. The do-nothing reference is the target decoding autoregressively on the distracted prompt. It fixes the accuracy-under-attack floor and the 1.0× speed reference. For Qwen3-8B on attacked AIME, Distractor Injection Attacks reports 4.9% accuracy, and we will reproduce this number in our harness before comparing anything.

Vanilla speculative decoding. Token-level speculation with the same drafter is the speed reference. On GSM8K, Lookahead Reasoning reports 1.4× for it on clean inputs. It preserves the target's distracted distribution, so its accuracy should equal the plain-decoding floor.

SpecReason. The strongest speed competitor is SpecReason. It reports 1.4×–3.0× speedup and +0.4% to +9.0% accuracy on clean AIME, MATH500, and GPQA Diamond. We run it with its published thresholds and also re-tuned thresholds, using the same drafter, same target, and same step delimiter. For SpecReason or any speculative method on distractor-injected prompts, there is no reported result found on Lacuna, so we produce that number ourselves.

SFT+DPO defense. The strongest robustness competitor is the SFT+DPO defense from Distractor Injection Attacks. It raises Qwen3-8B's attacked AIME accuracy from 4.9% to 57.8%. We train it on the same distractor sources we use for probe training, with a compute budget reported alongside ours. Our probe training is orders of magnitude cheaper, so we report the comparison both raw and per unit of training compute.

Steered Speculative Decoding. SD² is included to test whether target-state steering helps or hurts under distraction.

Ablations. Ablations isolate the two ingredients: drafter masking with SpecReason's original acceptance, and probe-gated acceptance with an unmasked drafter.

What counts as beating the baselines. Beating SpecReason means three things together: higher attacked accuracy, equal or better speedup, and clean accuracy within one point. Beating SFT+DPO on accuracy is not required. Matching a substantial fraction of its recovery with no target retraining, while also being faster, is the bar. If our method composed with a DPO-trained target exceeds DPO alone, that counts too.

Why not just…?

Why not just filter the prompt once and run ordinary speculation on the clean version? This is the most serious alternative, and it may work well when distractors are easy to spot. Its weakness is that a hard filter is irreversible. Any relevant sentence it removes is gone for both models, and the "summarize instead of read" gains reported in Less is More and UDA rely on the filter being right. In our design the target still sees the full prompt and can override a wrongly masked drafter. We will run exactly this baseline: the same relevance scorer, hard pruning, and vanilla speculation. The decoy-prompt set is where we expect it to fail. If it matches us on decoys too, the probe-gated acceptance is unnecessary and we will report that.

Why not just retrain the target with SFT and DPO? Retraining works: 4.9% to 57.8% on attacked AIME. But it requires curated preference data and full fine-tuning of every model served, and it offers no speedup. Breaking Focus and the distractor paper both flag training overhead as a limitation. The honest framing is that our method is the training-free-for-the-target option. The composition experiment tests whether the two are complementary.

Why not just ask the verifier, in text, whether each step is relevant to the original question? A text relevance judge is a natural extension of SpecReason's scoring prompt, and we will run it. Two pieces of evidence suggest it is weaker. Covert compliance means the target already treats the side-quest as legitimate work. And Lookahead Reasoning reports that text-judge verification costs non-trivial compute. Internal probes have repeatedly outperformed text-based classifiers on the same question: 60.0% versus 53.5% for a BERT text baseline in Knowing Before Saying, and stronger than GPT-5.4-mini in Speculative Probing. If the text judge matches the probe's accuracy at similar overhead, the internal-state ingredient is not pulling its weight.

Why not just tell the model to ignore irrelevant context? Breaking Focus reports that such instructions fail against adaptive distractors. We include it as a cheap row to confirm this in the reasoning-model setting.

Decisive Checks

Clean inputs must not suffer. On clean AIME, MATH500, GSM8K, and GPQA Diamond, focused speculation must stay within one point of target accuracy and match SpecReason's speedup. Otherwise the method is a robustness patch that taxes normal use.

Probe generalization. We train the step-relevance probe only on math side-quests and test on ZebraLogic, LiveCodeBench-Pro, and Breaking Focus tree-search distractors. Breaking Focus reports GPT-4o-mini dropping from 85.7% to 22.0% on CommonsenseQA with these distractors. If probe accuracy, measured as AUROC against held-out step labels, falls sharply off-distribution, the method is only a learned defense against known attacks. We would then reframe it as such.

Tokens and speed. We count tokens spent on the distractor and wall-clock time per solved problem. The mechanism predicts that both fall together under attack: rejected side-quest steps cost one scoring pass, while generated side-quest steps cost hundreds of target tokens. If speed improves but side-quest tokens do not fall, the gains come from ordinary speculation, not from focus.

Misprobing harm. On MuSiQue, whose adversarially chosen distractor paragraphs sit beside required bridge facts, we measure how often the target overrides a wrongly masked drafter. The original MuSiQue paper reports a best-model F1 of 47.3 against 78.0 for humans (MuSiQue), so there is room both to help and to hurt.