Instruct-of-Reflection: Enhancing Large Language Models Iterative Reflection Capabilities via Dynamic-Meta Instruction

Liping Liu, Chunhong Zhang, Likang Wu, Chuang Zhao, Zheng Hu, Ming He, Jianping Fan

Abstract

Self-reflection for Large Language Models (LLMs) has gained significant attention. Existing approaches involve models iterating and improving their previous responses based on LLMs' internal reflection ability or external feedback. However, recent research has raised doubts about whether intrinsic self-correction without external feedback may even degrade performance. Based on our empirical evidence, we find that current static reflection methods may lead to redundant, drift, and stubborn issues. To mitigate this, we introduce Instruct-of-Reflection (IoRT), a novel and general reflection framework that leverages dynamic-meta instruction to enhance the iterative reflection capability of LLMs. Specifically, we propose the instructor driven by the meta-thoughts and self-consistency classifier, generates various instructions, including refresh, stop, and select, to guide the next reflection iteration. Our experiments demonstrate that IoRT achieves an average improvement of 10.1% over established baselines in mathematical and commonsense reasoning tasks, highlighting its efficacy and applicability.

Introduction

While Large Language Models (LLMs) have demonstrated remarkable capabilities in generating complex reasoning chains, their ability to iteratively refine these chains—a process known as self-reflection—remains surprisingly brittle. Recent empirical studies suggest that without "oracle" labels or external feedback, standard self-correction often fails to improve, and sometimes even degrades, model performance.

This paper identifies three primary failure modes in current "static" reflection strategies:

  1. Redundancy: The model repeats a correct answer unnecessarily, wasting computation.
  2. Drift: The model changes a correct initial answer into an incorrect one during reflection.
  3. Stubbornness: The model persists in an error despite being prompted to re-evaluate its logic.
Failure modes of static reflection: Redundant, Drift, and Stubborn.

To address these issues, we propose the Instruct-of-Reflection (IoRT) framework, which moves beyond simple self-correction prompts toward a dynamic, instructor-led iterative process.

The IoRT Framework

The IoRT framework is structured into three distinct stages designed to provide strategic, high-level guidance rather than simple iterative nudges.

Stage 1: Generate Meta-Thoughts

Instead of immediate problem-solving, a Meta-Thinker module utilizes few-shot learning to generate abstract, high-level strategies (e.g., "To solve this, first identify the historical timeline of X and compare it to Y"). These "meta-thoughts" serve as a strategic blueprint and are stored in a Meta Memory Module for retrieval during the iteration process.

Stage 2: Refresh and Self-Reflect

The system generates an initial response from a black-box LLM. A specialized Reflector module then evaluates this response based on plausibility and logical consistency, providing granular feedback for the next phase.

Stage 3: Instruct the Iteration

This is the core innovation of IoRT. An Instructor module, guided by the previously generated meta-thoughts and a self-consistency classifier, determines the optimal next step. Unlike static methods that always proceed to the next iteration, the Instructor can issue three types of dynamic instructions:

  • Stop: If the initial and reflected answers are consistent and logically sound, the process halts to conserve resources.
  • Select: If answers are inconsistent, the Instructor uses meta-thoughts to select the more logically robust reasoning chain.
  • Refresh: If both versions are fundamentally flawed, the Instructor triggers a "hard reset," forcing the model to generate a completely new approach to break cycles of stubbornness.
The IoRT framework architecture, illustrating the three stages of iterative reflection.

Experimental Evaluation

We evaluated IoRT across three challenging benchmarks: GSM8K (grade school math), SVAMP (linguistically varied math problems), and StrategyQA (multi-step commonsense reasoning). The framework was tested using both proprietary models (GPT-3.5, GPT-4) and open-source models (Llama-2 7B, 13B, and 70B).

Key Results

  • Performance Gains: IoRT achieved an average improvement of 10.1% over established baselines. On GSM8K, IoRT with GPT-3.5 reached 84.6% accuracy, significantly outperforming standard Chain-of-Thought (76.6%) and the CRITIC framework (77.3%).
  • Efficiency: By utilizing the "Stop" instruction, IoRT reduced average call overhead by approximately 27.6% compared to fixed-iteration reflection methods. GPT models, in particular, reached high-quality solutions faster, requiring 1.9 fewer calls on average than Llama-2 models.
  • Stability: Larger models like GPT-4 demonstrated significantly higher stability (94.3% stable iterations) compared to smaller models like Llama-2 7B, which were more prone to performance drift.

Discussion and Conclusion

The success of IoRT highlights that effective self-reflection in LLMs requires more than just internal checking; it requires strategic, meta-level instruction. By acting as a teacher that provides high-level principles and knows when to stop or reset, the IoRT framework successfully mitigates the common pitfalls of drift and stubbornness.

Future work will focus on closing the gap between open-source and proprietary models. While IoRT significantly improves Llama-2 performance, smaller models still struggle to generate the high-quality meta-thoughts necessary for the first stage. We suggest exploring knowledge distillation to help smaller models internalize these strategic capabilities.

Summary updated

Abstract

Self-reflection for Large Language Models (LLMs) has gained significant attention. Existing approaches involve models iterating and improving their previous responses based on LLMs' internal reflection ability or external feedback. However, recent research has raised doubts about whether intrinsic self-correction without external feedback may even degrade performance. Based on our empirical evidence, we find that current static reflection methods may lead to redundant, drift, and stubborn issues. To mitigate this, we introduce Instruct-of-Reflection (IoRT), a novel and general reflection framework that leverages dynamic-meta instruction to enhance the iterative reflection capability of LLMs. Specifically, we propose the instructor driven by the meta-thoughts and self-consistency classifier, generates various instructions, including refresh, stop, and select, to guide the next reflection iteration. Our experiments demonstrate that IoRT achieves an average improvement of 10.1% over established baselines in mathematical and commonsense reasoning tasks, highlighting its efficacy and applicability.
BibTeX
@inproceedings{liu2025instruct,
  author = {Liping Liu and Chunhong Zhang and Likang Wu and Chuang Zhao and Zheng Hu and Ming He and Jianping Fan},
  title = {{Instruct-of-Reflection: Enhancing Large Language Models Iterative Reflection Capabilities via Dynamic-Meta Instruction}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2025},
  url = {https://openreview.net/forum?id=sP6KmfYKfy},
}
Figure