Recursive Think-Answer Process for LLMs and VLMs

Byung-Kwan Lee, Youngchae Chee, Yong Man Ro

TL;DR

The Recursive Think-Answer Process (R-TAP) introduces a confidence-guided, iterative reasoning framework that allows large language models (LLMs) and vision-language models (VLMs) to self-correct their mistakes across multiple reasoning cycles. By utilizing a dedicated confidence generator during reinforcement learning, R-TAP significantly boosts accuracy on complex tasks while paradoxically reducing inference-time rambling and latency—all with zero additional computational overhead at runtime.

What Problem Does This Paper Address?

State-of-the-art "Think-Answer" reasoners like DeepSeek-R1 and OpenAI's o1 have revolutionized AI problem-solving by separating internal monologue reasoning from final answer generation. However, these models typically rely on a single-pass reasoning trajectory.

If a model makes an early misstep or an "off-by-one" error, it remains vulnerable to outputting an incorrect final answer. Even when models generate self-reflective cues like "Oops! Let me try again," they often fail to systematically backtrack and fix the root error before the single pass concludes. Current reinforcement learning frameworks (like GRPO) optimize only single trajectories, leaving models unable to internally assess their own uncertainty and dynamically re-engage their reasoning cycles.

Methods

The authors propose R-TAP (Recursive Think-Answer Process), which trains models to evaluate their own certainty and iteratively refine their answers over $T$ recursive steps.

[Question] ---> (Base LLM/VLM) ---> [Think-Answer o(t)] 
                                           |
                                           v
[Final Answer] <--- (High Conf?) <--- [Confidence Generator C_𝜙]
       ^                                   |
       |________(Low Confidence)___________|

R-TAP introduces two key components:

  1. Confidence Generator ($C_\phi$): A module initialized from the pretrained model and fine-tuned via supervised learning to output a scalar confidence score (0 to 1) predicting the correctness of a given reasoning path. Crucially, this generator is discarded at inference time, ensuring zero runtime overhead.
  2. Recursive Reward Structure in GRPO: During reinforcement learning, the model is optimized using two novel confidence-driven rewards:
  3. Recursively Confidence Increase Reward ($R_{\text{Increase}}$): Rewards the model when its internal confidence score increases from one reasoning cycle to the next.
  4. Final Answer Confidence Reward ($R_{\text{Final}}$): Penalizes low-confidence final predictions, pushing the model to terminate only when it is highly certain.

Datasets

The framework was evaluated across a massive suite of competitive benchmarks:

  • LLM Benchmarks (Math & Code): AIME24, AIME25, HMMT Feb 25, OmniMath, GPQA Diamond, LiveCodeBench (v5/v6), GSM8K, MATH500, Minerva, and OlympiadBench.
  • VLM Benchmarks (Multimodal Math & Science): MMMU, MMMU-Pro, MathVista, MathVision, OlympiadBench, MathVerse, WeMath, and HallusionBench.

Training data included open-source reasoning sets such as Open-R1-Math (220K), codeforce-cot (100K), and PRIME (481K).

Key Insights and Ideas

  • The "Oops" Paradox: You might expect an iterative model to generate more self-reflective "Oops"-style text. Instead, the authors found that R-TAP-trained models exhibit significantly fewer self-reflective patterns at inference time. By learning to stabilize their internal reasoning during training, the models produce cleaner, more direct reasoning paths at runtime.
  • Faster Inference via Stability: Because the models make fewer reasoning errors and experience fewer circular "stalls," the overall inference time (measured in log-hours across benchmarks) actually decreased compared to baseline single-pass models.
  • Implicit Self-Correction: Because the confidence generator is removed during deployment, the model implicitly internalizes the capability to double-check its work and avoid pitfalls without needing an external verifier or extra parameters.

Key Results

R-TAP yielded dramatic performance improvements across both text and vision modalities:

  • Large Language Models: When applied to Phi-4-reasoning-plus, R-TAP boosted the model's accuracy on the grueling AIME25 benchmark from 78.0% to 83.7%, and on GPQA from 69.3% to 76.7%, narrowing the gap to closed-source frontiers like OpenAI o1.
Figure 1. Overall accuracy (%) of numerous large language models (LLMs) on five evaluation benchmarks—AIME25 [81], HMMT Feb 25 [2], OmniMath [29], GPQA [25], and LiveCodeBench [43].
  • Vision-Language Models: On the multimodal MM-Eureka-32B, R-TAP elevated MathVista performance from 74.8% to 80.2% and WeMath from 73.4% to 79.3%.
Figure 2. Overall accuracy (%) of numerous vision language models (VLMs) on five evaluation benchmarks—MMMU [110], Math- Vista [74], OlympiadBench [36], MathVision [94], and MMMU-Pro [111].

A qualitative example highlights how the model successfully catches and fixes an off-by-one error over three distinct internal cycles before delivering the correct mathematical answer:

Figure 3. Qualitative example of recursive think–answer process on a combinatorics question. The model iteratively refines its solution across multiple reasoning cycles, successfully correcting initial misconceptions such as off-by-one errors.

Limitations and Future Work

The primary limitation of R-TAP lies in its training efficiency. Due to the batch-dependent nature of current deep learning hardware and frameworks, all $T$ recursive trajectories must be generated in parallel during the training phase, even if a highly confident answer emerges early in the sequence. This introduces notable computational and memory overhead during training.

The authors note that developing more dynamic, hardware-friendly parallelization strategies for early-stopping during training remains an open and promising direction for future research.

Summary updated

Abstract

Think-Answer reasoners such as DeepSeek-R1 have made notable progress by leveraging interpretable internal reasoning. However, despite the frequent presence of self-reflective cues like "Oops!", they remain vulnerable to output errors during single-pass inference. To address this limitation, we propose an efficient Recursive Think-Answer Process (R-TAP) that enables models to engage in iterative reasoning cycles and generate more accurate answers, going beyond conventional single-pass approaches. Central to this approach is a confidence generator that evaluates the certainty of model responses and guides subsequent improvements. By incorporating two complementary rewards—Recursively Confidence Increase Reward and Final Answer Confidence Reward—we show that R-TAP-enhanced models consistently outperform conventional single-pass methods for both large language models (LLMs) and vision-language models (VLMs). Moreover, by analyzing the frequency of "Oops"-like expressions in model responses, we find that R-TAP-applied models exhibit significantly fewer self-reflective patterns, resulting in more stable and faster inference-time reasoning. We hope R-TAP pave the way evolving into efficient and elaborated methods to refine the reasoning processes of future AI.
BibTeX
@inproceedings{lee2026recursive,
  author = {Byung-Kwan Lee and Youngchae Chee and Yong Man Ro},
  title = {{Recursive Think-Answer Process for LLMs and VLMs}},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year = {2026},
  url = {https://openreview.net/forum?id=28zV0FxV6k},
}

Loading related content...

Figure