Partial Reasoning in Language Models: Search and Refinement Guided by Uncertainty

Murilo da Luz, Bruno Brandão, Luana Martins, Gustavo Oliveira, Bryan de Oliveira, Luckeciano Melo, Telma SoaresarXiv 2026 · DOI · arXiv

TL;DR

PREGU (Partial Reasoning Guided by Uncertainty) is a novel framework that improves LLM reasoning by monitoring Shannon entropy during text generation. When the model becomes "uncertain" (high entropy), PREGU halts the process and triggers a targeted search in the latent space to refine the reasoning path, effectively mimicking the human "System 2" analytical thinking process.

What Problem Does This Paper Address?

Large Language Models (LLMs) are surprisingly good at reasoning, but they often stumble on complex, multi-step problems like advanced math or strategic planning. This is largely because they generate text "greedily"—one token at a time—without a built-in mechanism to simulate long-term outcomes or explore alternative paths. Once a model makes a slight logical error early in a chain, the mistake cascades.

Existing techniques like "Soft Reasoning" (SR) try to fix this by searching through the model's internal mathematical representations (latent space) rather than just the text. However, SR typically optimizes only from the very beginning of a prompt. This is computationally expensive and inefficient because it spends effort on parts of the sequence where the model might already be confident. The authors of this paper ask: Can we detect exactly when a model is confused and only apply intensive search at those specific "critical junctures"?

Methods

The authors introduce PREGU, a two-stage process that integrates uncertainty detection with latent-space refinement:

  1. Stage 1: Partial Reasoning (Token Space): The model begins generating multiple reasoning paths. At every step, PREGU calculates the Shannon Entropy of the output distribution. If the entropy exceeds a specific threshold ($\tau$), it signals that the model is "unsure" which token to pick next. The generation is immediately halted, creating a "partial reasoning prefix."
  2. Stage 2: Refinement (Latent Space): Instead of starting over, PREGU treats these partial paths as roots. It then uses the Soft Reasoning method to perform a localized search. This involves applying controlled Gaussian perturbations to the embedding of the first token and using Bayesian Optimization to find a version of that embedding that leads to the most coherent and correct final answer.

The quality of these refined paths is judged by a reward function that balances "correctness" (verified by the LLM itself) and "coherence" (syntactic fluency).

Datasets

The researchers evaluated PREGU using three popular open-source models (LLaMA-3-8B, Mistral-7B, and Qwen2-7B) across four challenging reasoning benchmarks:

  • GSM8K: Grade-school math word problems.
  • GSM-Hard: A more difficult version of GSM8K requiring deeper compositional inference.
  • SVAMP: Math problems with superficial structural variations to test robustness.
  • StrategyQA: Questions requiring multi-step commonsense reasoning.

Key Insights and Ideas

  • Entropy as a "Metacognitive" Signal: The most striking finding is that entropy spikes aren't random. High entropy frequently occurs at logical transitions—words like "For", "Since", and "Given". These are the "forks in the road" where the model must choose a specific logical direction.
  • System 1 vs. System 2: The authors frame PREGU as a computational version of Dual Process Theory. The fast, intuitive generation is "System 1," while the entropy-triggered latent search acts as the slow, analytical "System 2."
  • Efficiency through Localization: By only triggering search when uncertainty is high, PREGU avoids wasting compute on "easy" parts of the reasoning chain, making the search more focused and effective than standard Soft Reasoning.

Key Results

PREGU consistently outperformed or matched the original Soft Reasoning method and other baselines like Chain-of-Thought (CoT) and Self-Consistency (SC).

  • GSM-Hard Gains: On the toughest math benchmark, PREGU showed significant robustness. For example, with LLaMA-3-8B, PREGU achieved 35.2% accuracy, a notable jump from Soft Reasoning's 28.2%.
  • Consistency: Across all models (LLaMA, Mistral, Qwen), PREGU showed higher accuracy on StrategyQA and GSM-Hard compared to standard prompting, suggesting that "stopping to think" is particularly beneficial for long-form logical chains.
  • Linguistic Correlation: The paper includes a word cloud (Figure 2) showing that the highest entropy tokens are almost exclusively discourse connectives, proving that uncertainty is tied to the structural logic of the answer.

Limitations and Future Work

While promising, PREGU has a few hurdles to clear:

  • The Threshold Problem: The entropy threshold ($\tau$) is currently a fixed hyperparameter. If it's set too high, the model never "stops to think"; too low, and the reasoning becomes fragmented. Future work aims to make this threshold dynamic based on the problem's complexity.
  • Self-Verification Bias: The system relies on the LLM to verify its own answers during the search phase. If the model is fundamentally "blind" to a specific type of error, the reward function might lead it astray.
  • Search Scope: Currently, the search only optimizes the very first token's embedding. The authors suggest that expanding this search "window" to multiple embeddings could lead to even smoother and more contextual refinements.

Summary updated

Abstract

The use of Large Language Models (LLMs) for reasoning and planning tasks has drawn increasing attention in Artificial Intelligence research. Despite their remarkable progress, these models still exhibit limitations in multi-step inference scenarios, particularly in mathematical and logical reasoning. We introduce PREGU (Partial Reasoning Guided by Uncertainty). PREGU monitors the entropy of the output distribution during autoregressive generation and halts the process whenever entropy exceeds a defined threshold, signaling uncertainty. From that point, a localized search is performed in the latent space to refine the partial reasoning and select the most coherent answer, using the Soft Reasoning method. Experiments conducted with LLaMA-3-8B, Mistral-7B, and Qwen2-7B across four reasoning benchmarks (GSM8K, GSM-Hard, SVAMP, and StrategyQA) showed performance greater than or similar to Soft Reasoning, indicating that entropy can serve as an effective signal to trigger selective refinement during reasoning.
BibTeX
@misc{luz2026partial,
  author = {Murilo da Luz and Bruno Brandão and Luana Martins and Gustavo Oliveira and Bryan de Oliveira and Luckeciano Melo and Telma Soares},
  title = {{Partial Reasoning in Language Models: Search and Refinement Guided by Uncertainty}},
  year = {2026},
  eprint = {2601.12040},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2601.12040},
}

Loading related content...

Figure