Think in Parallel, Answer as One: Logit Averaging for Open-Ended Reasoning

Haonan Wang, Chao Du, Kenji Kawaguchi, Tianyu Pang

TL;DR

THINKMERGE is a training-free decoding strategy that enables large language models (LLMs) to perform "parallel test-time scaling" for open-ended tasks. By running multiple reasoning paths simultaneously and averaging their next-token logits during the final answer phase, it brings the performance benefits of majority voting to complex tasks like coding and deep research, where traditional voting mechanisms fail.

The Challenge of Open-Ended Reasoning

Recent breakthroughs in reasoning models (such as OpenAI’s o1 and DeepSeek-R1) rely heavily on "test-time compute scaling"—giving the model more time to "think" before generating a final response. Currently, there are two primary ways to achieve this:

  1. Sequential Scaling: Extending the reasoning chain (e.g., prompting more self-corrections).
  2. Parallel Scaling: Generating multiple independent answers and using Majority Voting to select the best one.
Accuracy comparison across AIME and HMMT datasets showing Pass@1, Majority@8, and Pass@8 metrics.

While Majority Voting is incredibly effective for closed-ended problems like math or multiple-choice questions (where there is a single definitive answer), it struggles with open-ended reasoning. If you ask three models to write a complex Python script, they might all produce correct but syntactically distinct code. Because the output strings don't match exactly, you cannot simply "vote" on them. This paper introduces a method to capture the "wisdom of the crowd" at the logit level, allowing models to converge on a single, high-quality open-ended response.

Introducing THINKMERGE

The authors propose THINKMERGE, a plug-and-play decoding framework that operates in two distinct stages:

  1. Diverse Reasoning Generation: The model generates $K$ independent reasoning traces (the "thinking" phase) until it hits a predefined delimiter like </think>. By using high temperature settings during this phase, these $K$ paths explore diverse logical directions.
  2. Ensembled Answer Decoding: Once the thinking phase concludes, the model enters the "answer" phase. At every single token step, the system calculates the pre-softmax logits for all $K$ paths, averages them, and then applies a single softmax to choose the next token. This chosen token is then fed back into all $K$ contexts to maintain synchronization.

To ensure this process is efficient in production environments like vLLM or SGLang, the authors utilize "Bubble Squeezing" (left-padding reasoning traces to align them in the KV cache) or Flex-Attention to mask out padding tokens from shorter reasoning streams.

Key Insights

  • Logits > Probabilities: Averaging raw logits (Direct-Merge) proves significantly more robust than averaging normalized probabilities (Prob-Merge). Prob-Merge tends to "wash out" good answers if several paths are confidently wrong, whereas logit averaging maintains a stronger signal for the most likely correct tokens.
  • The "Shortest-K" Heuristic: In math tasks (like AIME), selecting the $K$ shortest reasoning traces often yields better results by avoiding "overthinking" loops. However, in coding, this bias is less effective, as shorter traces might omit necessary helper functions or scaffolding.
  • Capability Threshold: Ensembling is only beneficial if the base model is sufficiently capable. For very small models (e.g., 3B parameters), ensembling more paths can actually hurt performance—a "garbage in, garbage out" effect where weak trajectories dilute the few accurate ones.

Performance and Results

The researchers evaluated THINKMERGE across a wide spectrum of tasks, including closed-ended math (AIME 2024/2025, GPQA Diamond, HMMT), open-ended coding (LiveCodeBench v5), and agentic deep research (GAIA, BrowseComp, XbenchDeepSearch).

Pass@N performance scaling across GPQA, AIME, and LiveCodeBench for various models.
  • Coding Gains: On the "Hard" subset of LiveCodeBench, THINKMERGE improved the pass@1 rate of DeepCoder-14B-Preview by +8.28% and Qwen3-8B by +7.58%.
  • Deep Research: For agentic tasks, WebSailor-7B saw a massive +10.2% improvement on XbenchDeepSearch and +4.8% on GAIA by merging just 4 parallel trajectories.
  • Math Competitiveness: On AIME 2025, THINKMERGE matched or slightly outperformed traditional Majority Voting, proving it is a viable unified strategy for both closed and open-ended tasks.
  • Efficiency: As a training-free method that integrates with standard decoding controls (Top-p, Top-k, etc.), it serves as a seamless "drop-in" upgrade for existing LLM deployments.

Limitations and Future Work

  • Inference Cost: Running $K$ parallel traces naturally requires $K$ times the compute during the reasoning phase. While "Bubble Squeezing" mitigates some overhead, it remains more expensive than single-chain generation.
  • Synchronization Dependency: The method relies on models having a clear "think-then-answer" structure with a consistent delimiter.
  • Future Direction: The authors suggest exploring "dynamic K," where the model autonomously decides how many parallel paths it needs based on the prompt's difficulty.

Summary updated

Abstract

Majority voting has proven effective for close-ended question answering by aggregating parallel reasoning traces. However, it is not directly applicable to open-ended reasoning, such as code generation and web-based deep research, where a “majority” over complete solutions is ill-defined. We introduce THINKMERGE, a training-free, plug-and-play decoding strategy that runs K parallel reasoning traces and averages their next-token logits at synchronization points to produce a single coherent output. THINKMERGE integrates seamlessly with vLLM/SGLang and remains compatible with standard decoding techniques such as Top-p/Top-k. Empirically, it matches or surpasses majority voting on AIME and GPQA, while delivering consistent gains on open-ended coding tasks: on LiveCodeBench (hard), pass@1 improves by +8.28% for DeepCoder-14B-Preview and +7.58% for Qwen3-8B. Beyond code, we further show that THINKMERGE improves web-based deep-research agents (e.g., WebSailor-7B/32B) across GAIA, BrowseComp-en/zh, and XbenchDeepSearch. These results demonstrate that parallel test-time scaling can benefit open-ended reasoning without relying on voting over complete outputs.
BibTeX
@inproceedings{wang2026think,
  author = {Haonan Wang and Chao Du and Kenji Kawaguchi and Tianyu Pang},
  title = {{Think in Parallel, Answer as One: Logit Averaging for Open-Ended Reasoning}},
  booktitle = {International Conference on Learning Representations},
  year = {2026},
  url = {https://openreview.net/forum?id=hvit36Dyzl},
}
Figure