TL;DR
While long chain-of-thought (CoT) models appear to self-correct and backtrack, their final predictions are overwhelmingly dictated by the very first step of their reasoning trace. When an initial step goes wrong, models rarely recover—resulting in a ~40% drop in accuracy. Leveraging this vulnerability, the authors introduce an early pruning algorithm that scores candidate first steps with a process reward model (PRM) and discards weak trajectories early, matching standard 64-sample majority voting while slashing inference costs by up to 70%.
What Problem Does This Paper Address?
Modern reasoning models (such as DeepSeek-R1 and Claude 3.7 Sonnet) generate long reasoning traces filled with phrases like "Wait, let me rethink that" or "Alternatively...", leading to a widespread assumption that these systems actively self-correct mistakes.
However, long-chain reasoning comes at massive test-time computational costs and often suffers from "overthinking." This paper investigates where reasoning failures actually originate. Are models genuinely exploring hypotheses and correcting mid-stream errors, or are their reasoning trajectories fundamentally constrained from the start?
Methods
-
Positional Bias & Step-to-Conclusion Similarity: The authors segmented reasoning traces into distinct logical units (using both GPT-5 and lexical delimiters like "alternatively" or fixed token counts) and measured the cosine embedding similarity (MiniLM and SPLADE) between each intermediate step $t_i$ and the final conclusion $c_T$.
-
First-Step Ablations and Targeted Perturbations:
- They isolated the first step $t_1$, forced the model to conclude immediately ($c_1 = \mathcal{M}(p \text{ <think> } t_1 \text{ </think>})$), and measured how predictive $t_1$'s correctness was for the final answer.
- They took verified, correct conclusions, stripped answer markers to serve as the initial step $t'_1$, and injected minimal numerical perturbations (e.g., $a \pm 1$) to test whether models could detect and recover from early mistakes.
- Early Pruning via First-Step Filtering (TopM):
Instead of generating $N=64$ full reasoning paths for self-consistency (majority voting), the method samples $N=64$ candidate initial steps truncated to a fixed budget (len($t_1^n$) = 512 tokens). A lightweight Process Reward Model (
Qwen2.5-Math-PRM-7B) scores each initial step. Only the top $M$ (e.g., $M=16$) candidates are continued to full completion before applying majority voting.
Datasets
The authors evaluated models across five multi-step reasoning benchmarks across math, science, and coding:
- Mathematics: AIME 2024, AIME 2025, and HMMT (February 2025).
- Science: GPQA Diamond (graduate-level reasoning QA).
- Code Generation: LiveCodeBench (problems from Jan–Apr 2025).
No new benchmarks were introduced; all tests used publicly available, competitive evaluation sets.
Key Insights and Ideas
- The First Step Casts the Die: The semantic similarity between intermediate steps and the final answer is highest at step 1 and drops sharply immediately after. The very first reasoning step sets the trajectory; subsequent steps primarily elaborate on or search locally around this initial framing.
- Self-Correction Is Largely Overstated: If the first step was incorrect, the final accuracy dropped by an average of 40% across models (43% on challenging problems).
- Brittleness to Early Errors: In perturbation experiments, injecting a tiny discrepancy ($a \pm 1$) into an otherwise flawless reasoning setup caused massive performance collapses (e.g., DS-R1-Qwen-7B accuracy plunged from 94.8% to 28.5%).
- Overthinking Even When Correct: Models consumed just as many tokens completing a trace when their first step was already sufficient to answer correctly as they did when working from a flawed initial step.
Key Results
- Matching Majority Voting with >70% Cost Reduction: Selecting $M=16$ continuations from $N=64$ pruned starts matched or exceeded the performance of standard $N=64$ majority voting across all benchmarks while requiring less than 30% of the token compute ($\sim 0.28\times$ token budget).
- Outperforming Fixed-Budget Baselines: On AIME/HMMT/GPQA, TopM with $M=16$ consistently outperformed standard unpruned sampling with $N=16$ (e.g., Qwen3-8B gained +3.3% average accuracy over $N=16$, and +1.1% over full $N=64$).
- Generality Across Models: Trends held across multiple model families and sizes, including DeepSeek-R1-Distill (1.5B to 32B), Qwen3 (1.7B to 235B), Skywork-OR1-7B, Magistral-Small, GPT-OSS-20B, and Claude-3.7-Sonnet.
- Low Overhead: Scoring a 512-token prefix with a 7B PRM added minimal computational cost, translating directly into a $2.7\times$ to $3.7\times$ end-to-end wall-clock speedup compared to generating 64 full traces.
Limitations and Future Work
- Scope of Benchmarks: Evaluations focused strictly on structured STEM reasoning (math, code, science). Positional dynamics in open-ended or creative tasks remain unverified.
- Step Definition Heuristics: Although fixing length to 512 tokens proved effective in practice, arbitrary token budgets do not always correspond to complete logical steps across heterogeneous prompt types.
- Reliance on External Verifiers: Pruning requires a reliable reward model or verifier to evaluate short initial traces; poor reward model performance on specialized domains could undermine pruning precision.
Summary updated