Language models that play games usually think out loud before they act, and that thinking is slow. Speculative decoding speeds up generation by letting a small "draft" model write text that a large "target" model then checks in one parallel pass, keeping what it agrees with. The step-level versions of this idea check each drafted reasoning step with a critic that asks whether the step looks good. In a simultaneous-move game, where every player commits to an action without seeing the others' choices, the quality of an individual step is the wrong thing to check. The only thing that reaches the environment is the probability distribution over actions the agent finally commits to, called its mixed strategy. The expected payoff of a mixed strategy is a bilinear function of both players' strategies, so a small change in the committed distribution produces a small, boundable change in payoff, whatever the reasoning text looked like.
We propose a hierarchical speculative reasoner for LLM agents in simultaneous-move games. A small model drafts the whole reasoning trace, covering both what the opponent is likely to do and what to do in response. A larger model then reads that draft in a single forward pass and reports its own distribution over legal actions at the point where the action is written. The draft is accepted if the two distributions are within a tolerance set by the game's payoff range. Otherwise the next model up the hierarchy regenerates the trace. The result is lossy speculation with an explicit bound on how much exploitability it can add. We test it where exploitability can be computed exactly.
Core Claim
Our claim is that, for simultaneous-move decisions, testing drafted reasoning on the committed action distribution gives a better tradeoff between exploitability and wall-clock time than two alternatives: lossless token-level speculation, and step-level semantic verification of the kind used in SpecReason and Lookahead Reasoning. We expect this for two reasons. First, many drafted traces that a step critic would reject still lead the target to the same distribution, so they need not be rejected. Second, fluent but wrong steps that a critic would accept can shift that distribution, and the action-slot test catches them.
Let $p_D$ be the action distribution the target model produces after reading the drafted trace, and $p_T$ the distribution it would commit to. If every entry of the payoff matrix lies in $[-R, R]$, then against any opponent strategy $q$ the per-decision payoff difference is at most $2R\,\mathrm{TV}(p_D, p_T)$, where TV is the total-variation distance between the two distributions. Accepting a draft only when that distance is below $\delta$ therefore bounds the added exploitability over a horizon of $H$ decisions:
$$\mathrm{Expl}(\pi_{\text{spec}}) \le \mathrm{Expl}(\pi_{\text{target}}) + 2RH\delta .$$
The bound follows from Hölder's inequality on the bilinear payoff $p^\top A q$, applied once per decision. In a three-tier chain the tolerances add across tiers.
One caveat is central and must be tested. The guarantee holds relative to "the target reading the draft," not "the target reasoning on its own." Whether those two are close is an empirical question, and the experiments below measure it directly.
Relation to Prior Work
Two strands of prior work motivate the proposal. The first is simultaneous-move modeling. NN-CCE, described in Tree Search for Simultaneous Move Games via Equilibrium Approximation, runs a no-regret learner inside tree search so that averaged play converges to a coarse correlated equilibrium, a relaxed notion of stable joint play that is cheap to compute. Its value estimates are neural networks and it involves no language model. Strat-Reasoner trains LLMs to reason recursively about opponents. It restricts itself to two-player alternating games and reports that reasoning rollouts take 50–60% of training time. We keep the equilibrium-oriented evaluation of the first and the language-reasoning agent of the second. What we add is inference-time speculation designed around simultaneous commitment.
Kwiatkowska et al. show that serializing concurrent moves into turns gives the later player an information advantage it would not have in the real game. This constrains our design: we cannot borrow alternating-turn formulations.
The second strand is hierarchical speculative decoding, which supplies our machinery. Fast Inference via Hierarchical Speculative Decoding chains drafters of increasing size and chooses the chain by a shortest-path computation, assuming token acceptances are independent. We reuse the tiered structure. We replace token acceptance with a test on the action distribution, which sidesteps that independence assumption because the test fires once per decision.
SpecReason has a large model score each small-model reasoning step from 0 to 9. Scaling Speculative Decoding with Lookahead Reasoning checks whether a drafted step is semantically equivalent to what the target would write. SpecSearch filters drafted thoughts with a process reward model, meaning a learned per-step scorer. All three verify steps against a notion of correctness suited to single-agent math and code. All three are exposed to fluent but wrong steps, and the research file notes this failure mode for SpecSearch and SpecReason. We verify only what enters the game.
PipeSpec and Speculative Speculative Decoding overlap drafting with verification at the systems level. They are complementary: our test could sit inside either scheduler.
Research Program
We will build a three-tier reasoner using DeepSeek-R1-1.5B, a 7B Qwen model, and QwQ-32B, the size classes used by SpecReason and SpecSearch.
Each decision prompt gives the public state and legal actions and asks for a trace in two parts: a belief about the opponent's action, then a commitment written as a probability over actions. Because moves are simultaneous, the opponent-belief part does not depend on the agent's own move this round. The two parts can therefore be drafted in parallel, with no fan-out over candidate own actions, which an alternating-turn formulation would require.
Verification is a single prefill pass by the next tier. That tier reads the draft up to the action slot and returns its own action distribution over the legal action tokens. We compute the TV distance to the drafter's distribution and accept the draft if the distance is below $\delta$. If it is not, the next tier regenerates the trace, with ordinary token-level speculation from the tier below. The tolerance $\delta$ is expressed in payoff units, $2R\delta$, so a single setting has the same meaning across games.
The analysis has three parts. We will check the exploitability bound empirically. We will measure the "reading versus reasoning" gap, that is, TV distance between the target's distribution after reading a draft and after its own reasoning. And we will characterize which game states trigger escalation. Our conjecture is that escalation concentrates in states where the best response is nearly tied between actions.
Proposed Experiments
Testbeds are simultaneous-move games in OpenSpiel: Goofspiel at small card counts, where NashConv can be computed exactly, together with a suite of simultaneous normal-form matrix games. NashConv is OpenSpiel's measure of total distance from Nash equilibrium. We will also include simultaneous-reveal variants of Kuhn poker, built by having both players commit a bet at the same time, so that we can contrast with Strat-Reasoner's alternating setting.
The primary metric is NashConv of the induced LLM policy plotted against wall-clock time per decision at batch size 1. Secondary metrics are head-to-head payoff against the vanilla target and draft acceptance rate. For game-strength context, NN-CCE reports a 91% win rate against counterfactual regret minimization (CFR, a standard equilibrium-finding solver) in Goofspiel-6 (Tree Search for Simultaneous Move Games via Equilibrium Approximation). That is a non-LLM reference, not a baseline to beat.
Baselines
The naive reference points are the QwQ-32B target decoding its full trace with no speculation, and the same target asked to output an action with no reasoning. The second matters because it tells us whether reasoning improves play in these games at all. If no-reasoning NashConv matches full reasoning, there is nothing to speed up and the premise fails.
Lossless two-model speculative decoding (Leviathan et al.) with the 1.5B drafter gives exact target behavior at some speedup. Token-level hierarchical speculative decoding (Fast Inference via Hierarchical Speculative Decoding) over the same three tiers is the strongest lossless competitor. Its reported gain is 1.77× over vanilla decoding versus 1.58× for the best single drafter on LayerSkip-70B summarization.
The strongest lossy competitor is SpecReason (SpecReason), which reports 1.4× to 3.0× speedup over vanilla reasoning-model inference. Lookahead Reasoning, which reports speedup rising from 1.4× to 2.1× on GSM8K, is a second step-level competitor. We found no reported result on Lacuna for any speculative reasoning method on a game benchmark. We will therefore rerun both methods ourselves on identical prompts, models, and hardware, and sweep each method's acceptance threshold with the same number of tuning runs we give our $\delta$.
Beating them means strictly dominating their NashConv–latency curves on held-out games: equal or lower exploitability at matched latency across the sweep.
Two ablations isolate the key ingredient. The first keeps our pipeline but swaps the action-distribution test for SpecReason-style step scoring. The second keeps the action test but applies it to the small model's direct action with no trace. This separates the value of checking the committed distribution from the value of drafting reasoning at all.
Why not just…?
Why not just use SpecReason or Lookahead Reasoning as they are? They are the real competitors, and on games where reasoning steps rarely change the final action they may perform comparably. Our argument is that step critics spend the target's compute judging text that does not affect the payoff, and that they can be fooled by fluent errors. The step-scoring ablation settles this directly. If it matches our method's curve, the contribution collapses to the parallel drafting of the opponent belief, and we will report it that way.
Why not just distill or fine-tune the small model to play well, as Strat-Reasoner does with a 4B model? This is a genuine threat. Strat-Reasoner-4B reaches 94.04/90.47 on Kuhn poker, compared with 77.72/86.56 for GPT-5-mini (Strat-Reasoner). Those results come from alternating play, though, and the paper names simultaneous moves as unsolved. Distillation is also complementary: a better drafter simply raises our acceptance rate. We will include a small model fine-tuned on target traces both as a standalone baseline and as our drafter. If it alone matches the target's NashConv, speculation is unnecessary for these games.
Why not just call an equilibrium solver? For our small testbeds a solver would win. We use them precisely because exploitability is computable there. The target setting is games described in language, as in Cicero-style Diplomacy (diplomacy_cicero), where no solver applies. We make no claim beyond what the testbeds measure.
Why not just serialize moves into turns and reuse alternating-turn methods? Serialization gives the second mover information it would not have (Kwiatkowska et al.), and an alternating formulation forces opponent reasoning to fan out over the agent's own candidate moves. We will run a serialized-prompt variant and report both its NashConv and its extra drafting cost.
Decisive Checks
The proposal fails if any of four checks fails. First, reasoning must reduce NashConv relative to no-reasoning answers. Second, the "reading versus reasoning" TV gap must be small relative to $\delta$, or the bound is vacuous in practice. Third, the observed exploitability increase must stay within $2RH\delta$. Fourth, the action-distribution test must beat the step-scoring ablation at matched latency.
Success looks like acceptance rates above those of step-level verification together with NashConv indistinguishable from the vanilla target at a speedup exceeding the lossless hierarchical baseline.
References
- Tree Search for Simultaneous Move Games via Equilibrium Approximation
- Strat-Reasoner: Reinforcing Strategic Reasoning of LLMs in Multi-Agent Games
- PRISM-games 3.0: Stochastic Game Verification with Concurrency, Equilibria and Time
- Fast Inference via Hierarchical Speculative Decoding
- SpecReason: Fast and Accurate Inference-Time Compute via Speculative Reasoning
- Scaling Speculative Decoding with Lookahead Reasoning
- Accelerating Large Language Model Reasoning via Speculative Search
- PipeSpec: Breaking Stage Dependencies in Hierarchical LLM Decoding
- Speculative Speculative Decoding
- OpenSpiel: A Framework for Reinforcement Learning in Games
- Fast Inference from Transformers via Speculative Decoding
- Automatic verification of concurrent stochastic systems