On the Role of Temperature Sampling in Test-Time Scaling

Yuheng Wu, Azalia Mirhoseini, Thierry TambeSubmission to ICLR 2026 · Rejected

TL;DR

Generating more samples ($K$) at a fixed sampling temperature eventually hits a hard plateau, leaving challenging problems unsolved no matter how large $K$ becomes. This paper reveals that different sampling temperatures unlock distinct subsets of hard problems; by scaling across temperatures rather than just sample counts, base language models can match the reasoning performance of reinforcement-learning-tuned models without additional training.

What Problem Does This Paper Address?

Test-time scaling (TTS)—generating multiple reasoning candidates in parallel and selecting the best output—is a popular alternative to expensive reinforcement learning (RL) post-training. Standard TTS assumes that generating more reasoning traces ($K$) steadily improves coverage (Pass@$K$).

However, the authors find that Pass@$K$ quickly hits a ceiling: scaling $K$ up to 13,312 at a fixed temperature yields no valid accuracy gains on difficult benchmarks once reasoning traces are rigorously verified. The authors ask: Have we reached the fundamental limit of parallel test-time scaling, or are we simply searching the wrong dimension?

Methods

  1. Multi-Temperature Scaling ($+T$): Instead of allocating the entire sampling budget to a single default temperature (e.g., $T = 0.6$), the budget is distributed across multiple temperatures ($T \in [0.0, 1.2]$ in increments of 0.1).
  2. Trace Verification: To prevent lucky guesses from inflating scores on integer-answer math competitions (AIME), problems with low accuracy ($\le 3\%$) have their complete step-by-step reasoning verified by GPT-5 against human-written solutions.
  3. Efficient Two-Stage Multi-Temperature Voting (Early Exit): Evaluating 12 temperatures naively multiplies compute costs. The authors design an early-exit algorithm:
  4. Intra-temperature voting: For each active problem, traces are generated iteratively. Each temperature tallies its top answer until its plurality exceeds a confidence threshold ($\tau_{\text{intra}} = 0.8$).
  5. Cross-temperature voting: Once all temperatures are confident, their majority answers vote together. If unanimous/high consensus ($\tau_{\text{cross}} = 1.0$) is reached, the problem is classified as easy and exits early.
  6. Low temperatures ($0.1$–$0.3$) are safely pruned because their outputs are subsumed by higher temperatures.

Datasets

The authors evaluate on five diverse benchmarks spanning mathematics, coding, and theory-of-mind reasoning:

  • AIME 2024 & AIME 2025: 30 high-school competition math problems per year.
  • MATH500: 134 Level-5 (hardest tier) problems evaluated with SymPy.
  • LiveCodeBench v6: 175 competitive programming problems (AtCoder, LeetCode) evaluated against private test suites.
  • Hi-ToM: 200 long-context, multi-agent theory-of-mind / dynamic logical reasoning tasks evaluated via process-level belief tracking.

Note: No new datasets were introduced; existing benchmarks were augmented with stricter trace-level verification.

Key Insights and Ideas

  • Different temperatures explore different reasoning paths: Easy questions are solved across almost all temperatures. However, hard problems lie on the axes of cross-temperature correlation plots—solvable at one specific temperature (e.g., $T=0.7$) but completely inaccessible at others (e.g., $T=0.9$), even with thousands of samples.
  • Base models can rival RL models via TTS: Scaling temperature allows base models (e.g., Qwen3-4B) to reach a Pass@All of $73.3\%$ on AIME 2025, matching its RL-trained counterpart (Polaris-4B-Preview at $75.7\%$) without fine-tuning.
  • Entropy doesn't always signal correctness: While the model's token entropy is significantly lower for correct traces on easy/medium questions, this gap collapses on hard problems. Models do not reliably "know that they know" on tough problems, ruling out naive entropy filtering as a universal solution.
  • Scaling $K$ beyond a point promotes guessing, not reasoning: Scaling from $K = 1,024$ to $13,312$ appeared to yield new correct answers, but rigorous LLM judging revealed these were unfaithful or lucky guesses rather than sound derivations.

Key Results

  • Expanded Reasoning Boundary: Across all model sizes (Qwen3-0.6B to 8B), temperature scaling consistently improved Pass@All over single-temperature Pass@1,024:
  • AIME 2025: Qwen3-4B jumped from $60.0\%$ to $73.3\%$ (+13.3 points); Qwen3-0.6B improved from $20.0\%$ to $36.7\%$ (+16.7 points).
  • LiveCodeBench: Qwen3-8B gained +7.4 points ($32.6\% \to 40.0\%$).
  • Hi-ToM: Qwen3-1.7B jumped +18.0 points ($37.0\% \to 55.0\%$).
  • Compute Reduction via Early Exit:
  • On MATH500, the two-stage voting method cut compute overhead by $54.4\%$ for Qwen3-8B with zero drop in accuracy ($97.8\%$).
  • Across all benchmarks, the method achieved an average compute reduction between $26.9\%$ and $46.3\%$, preserving nearly all accuracy gains by quickly retiring easy problems.

Limitations and Future Work

  • Compute Multiplier: Searching over a grid of temperatures without early exit can require up to $12\times$ more rollouts than standard sampling.
  • "Impossible" Problems Persist: Certain problems remain at the origin (unsolved across all temperatures and sample sizes), indicating that temperature scaling cannot replace base model pre-training or specialized RL for out-of-distribution reasoning.
  • Verifier Dependency: In practice, fully exploiting the expanded reasoning boundary requires an external verifier or reward model capable of picking sparse, valid solutions from large pools.
  • Future Directions: Investigating dynamic temperature schedules within a single generation (token-level or step-level adaptation) to combine exploratory and deterministic decoding paths into individual rollouts.

Summary updated

Abstract

Large language models (LLMs) can improve reasoning at inference time through test-time scaling (TTS), where multiple reasoning traces are generated and the best one is selected. Prior work shows that increasing the number of samples $K$ steadily improves accuracy. In this paper, we demonstrate that this trend does not hold indefinitely: at large $K$, further scaling yields no gains, and certain hard questions remain unsolved regardless of the number of traces. Interestingly, we find that different sampling temperatures solve different subsets of problems, meaning single-temperature scaling explores only part of a model’s potential. We therefore propose scaling along the temperature dimension, which enlarges the reasoning boundary of LLMs. Temperature scaling enables base models to reach performance comparable to reinforcement learning (RL)-trained counterparts, without additional post-training. We further provide a comprehensive analysis of this phenomenon and design a multi-temperature voting method that reduces the overhead of temperature scaling. Overall, our findings suggest that TTS is more powerful than previously thought, and that temperature scaling offers a simple and effective way to unlock the latent potential of base models.
BibTeX
@inproceedings{wu2025role,
  author = {Yuheng Wu and Azalia Mirhoseini and Thierry Tambe},
  title = {{On the Role of Temperature Sampling in Test-Time Scaling}},
  booktitle = {ER @ NeurIPS},
  year = {2025},
  url = {https://openreview.net/forum?id=Q1VUsgstay},
}
Figure