A language model can spend extra compute at answer time by treating its answer as something to improve: score a draft with a reward model, then edit the draft so the score rises. One recent method does this with gradient descent on the model's continuous token scores, and it beats drawing many independent samples. Gradient descent, however, only moves locally. When a draft sits in a confident but wrong "basin", for example a derivation built on a bad early step, small smooth edits cannot leave it. Methods that resample whole chunks of text can make large moves but get no direction from the gradient.
We propose to combine the two in one sampler drawn from jump-diffusion modeling. A jump-diffusion is a random process that drifts continuously most of the time and occasionally makes sudden discrete jumps. Here the continuous part is gradient-guided editing of token scores. The jump part replaces one reasoning step with a fresh sample from the base model, and the swap is kept or rejected by a correctness-preserving accept/reject test. The new ingredient is the timing and placement of jumps. The jump rate rises when gradient progress stalls, and jumps land on the step the reward gradient blames. The sampler therefore spends compute on non-local moves only where local refinement has stopped paying off. The claim is narrow and testable. At matched compute, a stall-triggered and targeted jump schedule should beat gradient-only refinement, jump-only resampling, random restarts, and a fixed alternation of the two.
Core Claim and Mechanism
Fix a prompt $x$, a base model $p_\theta$, a reward model $r$, and a temperature $\beta$. The target is the reward-tilted distribution
$$ \pi(y) \propto p_\theta(y \mid x)\, \exp\!\big(r(x,y)/\beta\big), $$
where $y$ is the full reasoning chain. This is the same objective $\nabla$-Reasoner optimizes. That paper adds a KL-to-base fluency penalty to reward maximization and runs first-order descent on the token scores before the softmax, using a Gumbel-softmax straight-through estimator to pass gradients through discrete tokens. We reuse its local move almost unchanged, adding small injected noise so the move becomes a Langevin step (gradient plus Gaussian noise) rather than pure descent.
The jump move works on reasoning steps, meaning sentence or line blocks of the chain. At random times, the sampler picks a block $k$ and redraws that block from $p_\theta$ given the preceding text. The later blocks are kept, and the swap passes a Metropolis-Hastings test. Because the proposal is the base model itself, the base-model probability of the replaced block cancels from the acceptance ratio. What remains is
$$ \alpha = \min\!\Big(1,\; e^{(r(y') - r(y))/\beta}\, \frac{p_\theta(y_{>k} \mid y_{<k}, b'_k)}{p_\theta(y_{>k} \mid y_{<k}, b_k)}\Big). $$
Here $b_k$ and $b'_k$ are the old and new block, and $y_{>k}$ is the untouched suffix. The test costs one reward call and one forward pass, and it leaves $\pi$ invariant. This is how Jump-Diffusion Langevin Dynamics for Multimodal Posterior Sampling keeps exactness when it mixes Langevin steps with Poisson-timed jumps over Bayesian neural network weights. That paper supplies the method: it shows that gradient-only Langevin has very long escape times between isolated modes and that jumps shorten them.
Our departure from that work is that its jumps arrive at a fixed rate with a fixed proposal. Ours have a state-dependent intensity $\lambda_t = \lambda_0\,\sigma(\kappa(\tau - \Delta_t))$. In this formula, $\Delta_t$ is the reward gained by the last few gradient steps, $\tau$ is a stall threshold, $\kappa$ sets how sharply the rate switches, and $\sigma$ is the logistic function. The block to jump is drawn in proportion to how much of the reward-gradient norm falls on it. The sampler thus jumps where the reward model says the error lives, and only once smooth editing has stopped improving the score.
Relation to Prior Work
Prior work falls into three groups. The first motivates the proposal by showing that smooth or local test-time refinement gets trapped. Escaping Confidence Trap finds that diffusion language models fall into confident, repetitive, wrong basins. It escapes them with hand-built logit mutations on numbers and symbols, triggered when confidence drops below a threshold. Our stall trigger and gradient-targeted block choice replace those heuristics with a rule that needs no domain-specific token lists and preserves a known target distribution. $\nabla$-Reasoner itself falls back on rejection sampling when gradients stall, which we read as indirect evidence for the gap.
The second group supplies the jump half. Effective Test-Time Scaling of Discrete Diffusion through Iterative Refinement frames reward-guided refinement as Multiple-Try Metropolis: it repeatedly corrupts and resamples parts of a draft, proves convergence, and shows this beats Best-of-$N$. It has no gradient drift and uses a fixed perturbation schedule, so it is the "jumps only" end of our design space. Scaling Inference Time Compute for Diffusion Models shows for image diffusion that verifier-guided search over noise and paths beats simply adding denoising steps. That result constrains us: we do not expect more gradient steps alone to be the efficient use of compute.
The third group is the jump-diffusion literature proper. Trans-Dimensional Generative Modeling via Jump Diffusion Models uses jumps to insert and delete components during generation. Coevolutionary Continuous Discrete Diffusion couples a discrete token chain with a continuous embedding diffusion during training. Neither does verifier-guided search at test time, and we do not attempt their training-time changes.
Research Program
We will build the sampler on Qwen-2.5-7B-Instruct, the model $\nabla$-Reasoner reports on, with the same reward model and fluency weight. That choice makes its numbers directly comparable. Blocks are newline-delimited reasoning steps.
The work has three parts. The first is the sampler itself, with its accept/reject correction. The second is a compute accountant that counts every forward pass, backward pass, and reward call in forward-pass equivalents, with a backward pass costed at two forward passes; this unit is used for every method.
The third part is a trap diagnostic. For each problem, we run the gradient-only sampler and record whether it ends in a wrong-but-stable state: high reward, unchanged final answer over the last 25% of steps, and an incorrect answer. We then measure how often each method leaves such states, and how much compute that takes. This "escape time" is the quantity jump-diffusion theory predicts should shrink. It lets us check that any accuracy gain comes from escaping traps rather than from extra sampling diversity in general.
The hyperparameters $\lambda_0$, $\tau$, $\kappa$, and $\beta$ will be tuned on a held-out slice of MATH training problems. The tuning budget will be equal to the budget given to the baselines.
Baselines
The naive reference point is greedy decoding of the same model. The file gives this as 71.2% on MATH-500 for Qwen-2.5-7B-Instruct, as reported by $\nabla$-Reasoner. Next is Best-of-$N$ with the same reward model at budgets from $N=4$ to $N=32$. It is the simplest way to spend compute, and $\nabla$-Reasoner reports beating Best-of-8 with up to 40.2% fewer model calls.
The strongest existing method is $\nabla$-Reasoner itself, reported at 80.4% on MATH-500 with this model. It is the strongest competitor because it already exploits reward gradients, and our method is a strict extension of it. We will rerun it under our compute accountant rather than trust the published figure alone, so both methods share the reward model, prompts, and budget. Beating it means higher MATH-500 accuracy at equal forward-pass-equivalent budget, by more than the run-to-run spread across three seeds, at two or more budgets. Matching its accuracy with clearly less compute would also count.
The jump-only baseline is our sampler with the gradient drift switched off. It is an IterRef-style refinement for an autoregressive model, in the spirit of Effective Test-Time Scaling of Discrete Diffusion through Iterative Refinement. Three ablations isolate the key ingredient. The first uses a fixed Poisson jump rate, as in Jump-Diffusion Langevin Dynamics, with the rate tuned to the best constant. The second keeps the stall trigger but picks blocks uniformly. The third is a fixed alternation of $m$ gradient steps and one jump, with $m$ tuned.
On Countdown, no reported result was found on Lacuna for $\nabla$-Reasoner or Best-of-$N$ with this model. The file reports only a relative figure there: LaDiR's 30% absolute Pass@1 gain over an autoregressive baseline. All Countdown baselines will therefore be run fresh.
Why not just…?
Why not just restart $\nabla$-Reasoner from fresh samples when it stalls? A restart is a jump that throws away the whole chain, including correct early steps. A block jump keeps them, and the accept/reject test keeps the swap only if it does not lose much reward. We include random restarts with the same stall trigger and budget. If restarts match block jumps, then the "keep the good parts" argument is wrong, and the contribution shrinks to the stall trigger.
Why not just alternate $\nabla$-Reasoner with resampling on a fixed schedule? This is the real threat. A tuned fixed alternation may capture most of the benefit with none of the machinery. The fixed-alternation and fixed-rate ablations test exactly this. If they match our method at every budget, we will report that the gain comes from mixing local and non-local moves, not from adaptive timing, and we will drop the adaptive-rate claim.
Why not just add more noise or more gradient steps? Jump-Diffusion Langevin Dynamics shows that Langevin noise alone gives long escape times between separated modes. Scaling Inference Time Compute for Diffusion Models shows that adding denoising steps plateaus. Our temperature-swept Langevin-only run, scored with the trap diagnostic, checks whether this holds for reasoning chains.
Why not just use Evolutionary Decoding's mutations? Its mutations are hand-built for numbers and symbols in diffusion language models and keep no target distribution. The file reports 53.3% to 60.0% on AIME 2024 for LLaDA 2.0-mini with roughly 2–3x more decoding steps (Escaping Confidence Trap). We will port its number/symbol logit mutation as an alternative jump proposal inside our sampler. That isolates whether gradient-targeted base-model resampling beats the heuristic proposal on equal footing.
Proposed Experiments
The main experiment plots accuracy against compute on MATH-500, with AIME 2024/2025 and AMC 2023 as harder held-out sets. Budgets run from one to sixteen times the cost of greedy decoding, with three seeds each. Countdown at input sizes 4 to 8 serves as a combinatorial test. There, wrong partial plans form many isolated basins, so the jump component should matter most.
The first decisive check is whether the hybrid sampler's curve lies above both $\nabla$-Reasoner and jump-only at matched compute. The second is whether its advantage concentrates on problems the trap diagnostic flags as trapped for $\nabla$-Reasoner, with shorter measured escape times. If the gain is spread evenly instead, the mechanism story is wrong even if accuracy improves.
The third check guards against reward hacking, a known risk for both $\nabla$-Reasoner and verifier search in image diffusion. We report the gap between reward-model score and true answer accuracy for every method. A method that raises reward while accuracy stays flat counts as a failure, not a win.
References
- $\nabla$-Reasoner: LLM Reasoning via Test-Time Gradient Descent in Latent Space
- Jump-Diffusion Langevin Dynamics for Multimodal Posterior Sampling
- Escaping Confidence Trap: Evolutionary Decoding for Mathematical Reasoning in Diffusion LLMs
- Effective Test-Time Scaling of Discrete Diffusion through Iterative Refinement
- Scaling Inference Time Compute for Diffusion Models
- Trans-Dimensional Generative Modeling via Jump Diffusion Models
- Coevolutionary Continuous Discrete Diffusion: Make Your Diffusion Language Model a Latent Reasoner
- LaDiR: Latent Diffusion Enhances LLMs for Text Reasoning