Abstract
Test-time scaling has emerged as a powerful paradigm for enhancing the reasoning capabilities of Large Language Models (LLMs). While closed-ended tasks benefit from parallel scaling via majority voting, open-ended tasks like coding and deep research lack a clear consensus mechanism. We present THINKMERGE, a training-free decoding strategy that enables LLMs to generate multiple reasoning traces in parallel and synthesize a single response through logit averaging. Our results demonstrate significant performance gains across diverse benchmarks, including an 8.28% improvement in coding tasks, effectively extending the benefits of parallel computation to non-canonical domains.
Introduction: Scaling Reasoning at Test-Time
Recent breakthroughs in reasoning models, such as OpenAI’s o1 and DeepSeek-R1, rely on "test-time scaling"—allocating more computational resources to the model's "thinking" process before it delivers an answer. Currently, this scaling is achieved either sequentially (longer reasoning chains) or in parallel (multiple reasoning chains).
For closed-ended tasks with a single correct answer (e.g., mathematics), parallel scaling typically employs majority voting to select the most common output. However, this approach fails for open-ended tasks like software engineering or deep research, where multiple valid solutions may differ significantly at the character level. THINKMERGE addresses this gap, allowing open-ended reasoning to benefit from parallel computation without requiring a canonical answer.
Methodology: The THINKMERGE Framework
THINKMERGE is a two-stage, plug-and-play inference framework designed for high-throughput engines like vLLM and SGLang:
- Stage I (Map): The model generates $K$ independent, diverse reasoning traces (e.g., within
<think>tags) in parallel. High-temperature sampling ensures the exploration of varied logical paths. - Stage II (Reduce): Upon reaching the reasoning delimiter (e.g.,
</think>), the framework enters a joint decoding mode. At each step, it queries the model for next-token logits across all $K$ contexts. These logits are averaged, and a single token is sampled from the resulting distribution. This token is then synchronized back into all $K$ parallel contexts for the subsequent step.
This "soft voting" at the token level allows the model to synthesize a response reflecting the collective wisdom of its parallel thoughts. Variants like Shortest-K Merge and K Early-Ready further optimize for conciseness and reduced tail latency.
Evaluation and Benchmarks
The researchers evaluated THINKMERGE across a broad spectrum of reasoning tasks, ranging from competitive mathematics to agentic web research. Benchmarks included AIME 2024/2025, HMMT 2024/2025, GPQA Diamond, LiveCodeBench (v5), and agentic tasks like GAIA and XbenchDeepSearch.
Key Results
THINKMERGE demonstrates that logit-level merging is a robust unified strategy for both closed and open-ended reasoning:
- Coding Performance: On the LiveCodeBench (Hard) set, THINKMERGE achieved a +8.28% increase in the
pass@1rate for DeepCoder-14B and a +7.58% increase for Qwen3-8B compared to single-chain baselines. - Deep Research Agents: For the WebSailor-32B agent, merging 8 parallel trajectories resulted in a +10.2% point increase on XbenchDeepSearch.
- Mathematical Reasoning: On AIME and GPQA, the method matched or slightly outperformed traditional majority voting, proving its versatility.
Insights and Discussion
A critical finding is that averaging logits (pre-softmax) is significantly more robust than averaging probabilities (post-softmax), which often degrades performance as $K$ increases. Furthermore, the benefits of parallel scaling are subject to a capability threshold; while larger models like WebSailor-32B see massive gains, smaller models (e.g., 3B) can suffer from "garbage in, garbage out" if the parallel traces are of low quality.
Limitations and Future Work
While effective, THINKMERGE increases computational costs by a factor of $K$ during the reasoning phase. Future research will explore dynamic allocation of $K$ based on problem difficulty and more robust methods for trimming repetitive or degenerate reasoning suffixes. Additionally, refining the weighting of high-quality traces during the merge remains an open challenge for smaller, more fragile models.
Summary updated