Meta-Reasoner: Dynamic Guidance for Optimized Inference-time Reasoning in Large Language Models

Yuan Sui, Yufei He, Tri Cao, Simeng Han, Bryan Hooi

Abstract

Large Language Models (LLMs) increasingly rely on prolonged reasoning chains to solve complex tasks. However, this trial-and-error approach often leads to high computational overhead and error propagation, where early mistakes can derail subsequent steps. To address these issues, we introduce Meta-Reasoner, a framework that dynamically optimizes inference-time reasoning by enabling LLMs to "think about how to think." Drawing inspiration from human meta-cognition and dual-process theory, Meta-Reasoner operates as a strategic advisor, decoupling high-level guidance from step-by-step generation. It employs "contextual multi-armed bandits" to iteratively evaluate reasoning progress, and select optimal strategies (e.g., backtrack, clarify ambiguity, restart from scratch, or propose alternative approaches), and reallocates computational resources toward the most promising paths. Our evaluations on mathematical reasoning and puzzles highlight the potential of dynamic reasoning chains to overcome inherent challenges in the LLM reasoning process and also show promise in broader applications, offering a scalable and adaptable solution for reasoning-intensive tasks.

Introduction

While "scaling test-time compute"—giving models more time to "think," as seen in OpenAI’s o1—has led to massive performance gains, it faces two major hurdles: computational inefficiency and error propagation. Current LLMs often get stuck in "unproductive reasoning trajectories"—loops or dead ends where they continue to generate tokens even after a fundamental mistake has been made. Naively increasing the number of reasoning steps leads to diminishing returns because the model lacks a high-level "advisor" to tell it when a specific approach isn't working. This paper introduces a principled mechanism to holistically revise or redirect the ongoing reasoning process in real-time.

Methods: The Meta-Reasoner Framework

The Meta-Reasoner framework is inspired by dual-process theory in cognitive science, explicitly decoupling low-level stepwise generation (System 1) from high-level strategic selection (System 2).

The Meta-Reasoner architecture decouples low-level reasoning from high-level strategic guidance using a contextual multi-armed bandit approach.

The system operates in an iterative loop:

  1. Step-wise CoT Generation: The base LLM generates a partial reasoning step.
  2. Progress Reporting: A lightweight summarization function distills the recent reasoning into a concise report. This filters out granular noise, allowing the "meta-reasoner" to focus on high-level progress.
  3. Strategy Selection (The Bandit): The framework treats strategy selection as a Contextual Multi-Armed Bandit (CMAB) problem. Using the LinUCB algorithm, the system evaluates the "context" (the progress report) and selects the best "arm" (strategy) from a pool. This pool includes fixed actions like "Backtrack," "Restart," or "Continue," as well as dynamic strategies that can be refined on the fly.
  4. Strategic Guidance: The chosen strategy is fed back to the LLM to guide its next reasoning step.

By using a reward function that balances solution progress against resource usage (token cost), the bandit learns to find the most efficient path to a correct answer.

Key Results

Evaluations across mathematics (Game-of-24, AIME-2024), science (SciBench), and creative writing benchmarks demonstrate the effectiveness of this approach.

Performance comparison on reasoning benchmarks shows Meta-Reasoner consistently outperforming previous SOTA methods and matching specialized models like o1.
  • Accuracy Gains: Meta-Reasoner outperformed previous state-of-the-art methods like MACM and Reflexion by 9–12% across math and science tasks.
  • Efficiency and Speedup: The method reduced inference time by 28–35% under the same compute budget compared to other iterative reasoning frameworks. Notably, it achieved performance comparable to specialized models like o1-preview but with a 51–55% reduction in inference time.
  • Model Agnostic: The framework showed consistent improvements across various backbone models, including GPT-4o, Gemini-1206, and open-source models like Qwen3-8B and DeepSeek-R1-Distill.

Conclusion

Meta-Reasoner demonstrates that how we use compute is just as important as how much compute we use. By separating "doing" from "planning," the framework avoids the massive overhead of exhaustive search methods like Tree-of-Thoughts. While currently limited to text-based problems, the system's success across diverse domains suggests that meta-reasoning is a generalizable skill that can significantly enhance LLM performance in reasoning-intensive tasks.

Summary updated

Abstract

Large Language Models (LLMs) increasingly rely on prolonged reasoning chains to solve complex tasks. However, this trial-and-error approach often leads to high computational overhead and error propagation, where early mistakes can derail subsequent steps. To address these issues, we introduce Meta-Reasoner, a framework that dynamically optimizes inference-time reasoning by enabling LLMs to "think about how to think." Drawing inspiration from human meta-cognition and dual-process theory, Meta-Reasoner operates as a strategic advisor, decoupling high-level guidance from step-by-step generation. It employs "contextual multi-armed bandits" to iteratively evaluate reasoning progress, and select optimal strategies (e.g., backtrack, clarify ambiguity, restart from scratch, or propose alternative approaches), and reallocates computational resources toward the most promising paths. Our evaluations on mathematical reasoning and puzzles highlight the potential of dynamic reasoning chains to overcome inherent challenges in the LLM reasoning process and also show promise in broader applications, offering a scalable and adaptable solution for reasoning-intensive tasks.
BibTeX
@inproceedings{sui2026meta,
  author = {Yuan Sui and Yufei He and Tri Cao and Simeng Han and Bryan Hooi},
  title = {{Meta-Reasoner: Dynamic Guidance for Optimized Inference-time Reasoning in Large Language Models}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.649},
}
Figure