Verifier-Guided Self-Consistency: Test-Time Reasoning with Imperfect Generators and Verifiers

Ekaterina Fadeeva, Xueyan Li, Mrinmaya Sachan, Maxim PanovarXiv 2026 · arXiv

Large language models (LLMs) are increasingly capable of complex reasoning, but they remain "brittle." A single plausible-sounding but incorrect intermediate step can derail an entire chain-of-thought, leading to a wrong final answer. While we can use learned verifiers to check these steps, how exactly should we use that feedback during inference?

A new paper, "Verifier-Guided Self-Consistency: Test-Time Reasoning with Imperfect Generators and Verifiers," introduces a decoding framework called VGSC that aims to find the "Goldilocks zone" between diversity and correctness.

TL;DR

Verifier-Guided Self-Consistency (VGSC) is a stochastic decoding framework that uses both step-level correctness and future-outcome estimates to guide LLM reasoning. By combining these signals in a bounded rejection-resampling procedure, VGSC outperforms traditional beam search and post-hoc reranking methods while remaining robust to imperfect verifiers.

What Problem Does This Paper Address?

Improving reasoning at test time usually involves two signals: the generator (the LLM itself), which provides diversity, and the verifier (a separate model like a Process Reward Model), which guides correctness.

Existing methods typically lean too far in one direction:

  1. Post-hoc aggregation (e.g., Weighted Self-Consistency): You sample many complete paths and rank them at the end. This is diverse, but if the generator rarely produces a correct path, the verifier has nothing good to choose from.
  2. Verifier-guided search (e.g., Beam Search): You use the verifier to prune paths during generation. This can steer the model, but it often "over-concentrates" on narrow sets of trajectories. If the verifier makes a mistake (a false positive), the search gets trapped in a dead end, amplifying the error.

The authors argue that because both generators and verifiers are imperfect, we need a way to use verifier signals stochastically to maintain diversity while steering toward correctness.

Comparison of inference-time reasoning strategies. VGSC (right) balances generator support with verifier guidance through stochastic sampling.

Methods

VGSC treats reasoning as approximate sampling from a verifier-tilted trajectory distribution. The core idea is to sample next steps that are both likely under the base LLM and highly rated by the verifier.

The framework uses two complementary verifier signals:

  • Step Verifier ($V$): Estimates the immediate correctness of the current step.
  • Future-Outcome Predictor ($Q$): Estimates whether the current prefix is likely to lead to a correct final answer (continuation value).

The VGSC Algorithm: At each reasoning step, the LLM proposes a candidate next step. VGSC then uses a bounded rejection-resampling procedure:

  1. Propose a candidate step from the base LLM.
  2. Calculate an acceptance weight based on $V$ and $Q$.
  3. Accept the step with a certain probability or reject and try again (up to a cap $C$).
  4. If all $C$ proposals are rejected, fall back to a weighted selection from the candidates.

This process is repeated to generate multiple independent trajectories, which are finally aggregated using standard self-consistency (majority voting).

Datasets

The authors evaluated VGSC across five diverse reasoning benchmarks, summarized in the table below:

Table 1: Summary statistics of the evaluation datasets. Mean number of reasoning steps is computed for Qwen2.5 Math 1.5B Instruct using our rule-based step splitting procedure.

They used instruction-tuned models like Qwen2.5 Math 1.5B and Llama 3.2 3B as generators, paired with Qwen2.5 Math PRM 7B as the primary verifier.

Key Insights and Ideas

  • The Importance of $Q$: A step can be logically "correct" but "stagnant" (e.g., just restating the problem). The future-outcome predictor $Q$ is vital because it favors steps that actually move the needle toward a solution.
  • Stochasticity as a Safeguard: Unlike deterministic beam search, VGSC’s stochastic nature prevents it from collapsing into a single, potentially erroneous path. This makes it significantly more robust to "noisy" verifiers.
  • Adaptive Compute: Because it uses rejection sampling, VGSC is compute-efficient. If the LLM's first proposal is highly rated, the algorithm accepts it immediately and moves on, saving tokens compared to methods that always expand a full set of candidates.
A MATH example showing how $Q$ steers the model away from stagnant paths that $V$ alone might accept.

Key Results

VGSC consistently achieved the highest average accuracy across all benchmarks.

  • Accuracy Boost: On the MATH benchmark, VGSC ($\alpha=5$) achieved 81.2% accuracy, significantly higher than Beam Search (77.4%) and standard Self-Consistency (75.2%).
  • Efficiency: VGSC reduced the number of generated tokens by about 30% compared to Beam Search while achieving better results.
  • Robustness: In ablation studies where Gaussian noise was added to verifier scores, VGSC remained stable, whereas beam search performance degraded sharply as noise increased.
Table 2: Accuracy ↑across datasets and average generated tokens per sample for Qwen2.5 Math 1.5B Instruct generator with Qwen2.5 Math PRM 7B verifier. Results are averaged over 5 seeds. TFLOPs are estimated using Eq. (39);. The best result in each column is bold and the second-best result is underlined.

Limitations and Future Work

  • Auxiliary Model Training: VGSC requires a future-outcome predictor ($Q$). While the authors show this can be trained efficiently using Monte Carlo rollouts, it still adds training and engineering overhead compared to "plug-and-play" decoding.
  • Hyperparameter Tuning: The "verifier-trust" exponent ($\alpha$) is a sensitive hyperparameter. While a range of 10–20 worked well generally, the optimal value varies by task and verifier quality.
  • Verifier Alignment: VGSC still struggles if the verifier is fundamentally misaligned with the task (as seen in their SciQA failure case), suggesting that improving verifier quality remains a primary bottleneck.

Summary updated

Abstract

Large language models can solve many multi-step reasoning problems, but their reasoning remains brittle: plausible intermediate steps often lead to incorrect answers. Improving reasoning at test time requires balancing two imperfect signals: the generator, which provides diversity, and the verifier, which guides correctness. Existing approaches largely favor one side of this tradeoff. Verifier-weighted voting relies on trajectories sampled from the generator and cannot steer generation toward correct ones. By contrast, beam-style verifier-guided search can steer generation, but may over-concentrate on narrow trajectory sets and amplify verifier errors. We propose Verifier-Guided Self-Consistency (VGSC), a stochastic decoding framework that combines step-level correctness and future-outcome estimates to guide reasoning during generation. VGSC repeatedly proposes candidate next steps from the base language model and uses both signals in an acceptance/resampling procedure, while maintaining multiple trajectories for final answer-level self-consistency. We provide a probabilistic interpretation of VGSC as approximate sampling from a verifier-tilted trajectory distribution. Across multiple reasoning benchmarks, generator models, and verifier families, VGSC achieves higher average accuracy than representative baselines.
BibTeX
@misc{fadeeva2026verifier,
  author = {Ekaterina Fadeeva and Xueyan Li and Mrinmaya Sachan and Maxim Panov},
  title = {{Verifier-Guided Self-Consistency: Test-Time Reasoning with Imperfect Generators and Verifiers}},
  year = {2026},
  eprint = {2611.06606},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.06606},
}
Figure