TL;DR
THINKMERGE is a training-free decoding strategy that enables large language models (LLMs) to perform "parallel test-time scaling" for open-ended tasks. By running multiple reasoning paths simultaneously and averaging their next-token logits during the final answer phase, it brings the performance benefits of majority voting to complex tasks like coding and deep research, where traditional voting mechanisms fail.
The Challenge of Open-Ended Reasoning
Recent breakthroughs in reasoning models (such as OpenAI’s o1 and DeepSeek-R1) rely heavily on "test-time compute scaling"—giving the model more time to "think" before generating a final response. Currently, there are two primary ways to achieve this:
- Sequential Scaling: Extending the reasoning chain (e.g., prompting more self-corrections).
- Parallel Scaling: Generating multiple independent answers and using Majority Voting to select the best one.
While Majority Voting is incredibly effective for closed-ended problems like math or multiple-choice questions (where there is a single definitive answer), it struggles with open-ended reasoning. If you ask three models to write a complex Python script, they might all produce correct but syntactically distinct code. Because the output strings don't match exactly, you cannot simply "vote" on them. This paper introduces a method to capture the "wisdom of the crowd" at the logit level, allowing models to converge on a single, high-quality open-ended response.
Introducing THINKMERGE
The authors propose THINKMERGE, a plug-and-play decoding framework that operates in two distinct stages:
- Diverse Reasoning Generation: The model generates $K$ independent reasoning traces (the "thinking" phase) until it hits a predefined delimiter like
</think>. By using high temperature settings during this phase, these $K$ paths explore diverse logical directions. - Ensembled Answer Decoding: Once the thinking phase concludes, the model enters the "answer" phase. At every single token step, the system calculates the pre-softmax logits for all $K$ paths, averages them, and then applies a single softmax to choose the next token. This chosen token is then fed back into all $K$ contexts to maintain synchronization.
To ensure this process is efficient in production environments like vLLM or SGLang, the authors utilize "Bubble Squeezing" (left-padding reasoning traces to align them in the KV cache) or Flex-Attention to mask out padding tokens from shorter reasoning streams.
Key Insights
- Logits > Probabilities: Averaging raw logits (Direct-Merge) proves significantly more robust than averaging normalized probabilities (Prob-Merge). Prob-Merge tends to "wash out" good answers if several paths are confidently wrong, whereas logit averaging maintains a stronger signal for the most likely correct tokens.
- The "Shortest-K" Heuristic: In math tasks (like AIME), selecting the $K$ shortest reasoning traces often yields better results by avoiding "overthinking" loops. However, in coding, this bias is less effective, as shorter traces might omit necessary helper functions or scaffolding.
- Capability Threshold: Ensembling is only beneficial if the base model is sufficiently capable. For very small models (e.g., 3B parameters), ensembling more paths can actually hurt performance—a "garbage in, garbage out" effect where weak trajectories dilute the few accurate ones.
Performance and Results
The researchers evaluated THINKMERGE across a wide spectrum of tasks, including closed-ended math (AIME 2024/2025, GPQA Diamond, HMMT), open-ended coding (LiveCodeBench v5), and agentic deep research (GAIA, BrowseComp, XbenchDeepSearch).
- Coding Gains: On the "Hard" subset of LiveCodeBench, THINKMERGE improved the
pass@1rate of DeepCoder-14B-Preview by +8.28% and Qwen3-8B by +7.58%. - Deep Research: For agentic tasks, WebSailor-7B saw a massive +10.2% improvement on XbenchDeepSearch and +4.8% on GAIA by merging just 4 parallel trajectories.
- Math Competitiveness: On AIME 2025, THINKMERGE matched or slightly outperformed traditional Majority Voting, proving it is a viable unified strategy for both closed and open-ended tasks.
- Efficiency: As a training-free method that integrates with standard decoding controls (Top-p, Top-k, etc.), it serves as a seamless "drop-in" upgrade for existing LLM deployments.
Limitations and Future Work
- Inference Cost: Running $K$ parallel traces naturally requires $K$ times the compute during the reasoning phase. While "Bubble Squeezing" mitigates some overhead, it remains more expensive than single-chain generation.
- Synchronization Dependency: The method relies on models having a clear "think-then-answer" structure with a consistent delimiter.
- Future Direction: The authors suggest exploring "dynamic K," where the model autonomously decides how many parallel paths it needs based on the prompt's difficulty.
Summary updated