Instruct-of-Reflection: Enhancing Large Language Models Iterative Reflection Capabilities via Dynamic-Meta Instruction

Liping Liu, Chunhong Zhang, Likang Wu, Chuang Zhao, Zheng Hu, Ming He, Jianping Fan

Abstract

Self-reflection for Large Language Models (LLMs) has gained significant attention. Existing approaches involve models iterating and improving their previous responses based on LLMs’ internal reflection ability or external feedback. However, recent research has raised doubts about whether intrinsic self-correction without external feedback may even degrade performance. Based on our empirical evidence, we find that current static reflection methods may lead to redundant, drift, and stubborn issues. To mitigate this, we introduce Instruct-of-Reflection (IoRT), a novel and general reflection framework that leverages dynamic-meta instruction to enhance the iterative reflection capability of LLMs. Specifically, we propose the instructor driven by the meta-thoughts and self-consistency classifier, generates various instructions, including refresh, stop, and select, to guide the next reflection iteration. Our experiments demonstrate that IoRT achieves an average improvement of 10.1% over established baselines in mathematical and commonsense reasoning tasks, highlighting its efficacy and applicability.

Introduction: The Pitfalls of Static Self-Correction

While Large Language Models (LLMs) exhibit remarkable reasoning capabilities, their ability to autonomously correct errors remains a significant challenge. Traditional "static" reflection methods—where a model simply reviews its own output for a fixed number of iterations—often suffer from performance degradation. The authors identify three primary failure modes in these static processes:

  1. Redundancy: The model continues to reflect on and potentially alter an already correct answer, wasting computational resources.
  2. Drift: A model that initially provides a correct answer inadvertently "corrects" it into an incorrect one during subsequent iterations.
  3. Stubbornness: The model fails to recognize its own errors, persisting in a wrong answer despite multiple reflection attempts.
Figure 1: Illustration of the three failure modes in static reflection: Redundant, Drift, and Stubborn.

The IoRT Framework

To address these issues, the researchers developed Instruct-of-Reflection (IoRT). Unlike static methods, IoRT introduces a dynamic "Instructor" module that guides the model through a more structured, three-stage reasoning process.

Stage 1: Meta-Thought Generation

Before attempting a problem, the model generates "meta-thoughts"—high-level strategies and abstract principles relevant to the query. These thoughts act as a logical rubric and are stored in a "meta-memory" module for future retrieval, helping the model maintain a consistent reasoning path.

Stage 2: Refresh and Self-Reflect

The model generates an initial response and a subsequent reflective response. This stage provides the raw material for the Instructor to evaluate.

Stage 3: Dynamic Instruction

The core of IoRT is the Instructor, which utilizes meta-thoughts and a Self-Consistency Classifier to issue one of three specific instructions to manage the iteration:

  • Stop: If the responses are consistent and likely correct, the process halts to save time and cost.
  • Refresh: If the model appears "stubbornly" stuck on a wrong path, the Instructor triggers a fresh start with new perspectives.
  • Select: If the responses differ, the Instructor uses meta-thoughts to select the most logically sound answer, preventing "drift."
Figure 4: The three-stage workflow of the IoRT framework, highlighting the Meta-Thinker, Reflector, and Instructor modules.

Experimental Evaluation

The IoRT framework was tested across three major benchmarks: GSM8K (math), SVAMP (structured math), and StrategyQA (commonsense reasoning).

Key Results

  • Performance Gains: IoRT achieved an average accuracy improvement of 10.1% over standard baselines. Notably, it improved GPT-3.5's accuracy on GSM8K from 76.6% to 84.6%.
  • Efficiency: By utilizing the "Stop" instruction, IoRT reduced the average number of LLM calls by 27.6% compared to standard iterative reflection methods.
  • Model Stability: Smaller models, such as Llama-2-7B, showed significant reliability improvements, as the Instructor provided the necessary guardrails to prevent the high rates of "drift" typically seen in smaller architectures.

Conclusion and Future Work

IoRT demonstrates that effective self-correction requires more than just repetition; it requires high-level guidance. By introducing a dynamic Instructor and meta-reasoning, IoRT overcomes the inherent weaknesses of static reflection. Future research will focus on fine-tuning open-source models to enhance their native meta-thinking capabilities and extending the framework to specialized domains like legal and medical reasoning.

Summary updated

Abstract

Self-reflection for Large LanguageModels (LLMs) has gained significant attention. Existing approaches involve models iterating and improving their previous responses based on LLMs’ internal reflection ability or external feedback. However, recent research has raised doubts about whether intrinsic self-correction without external feedback may even degrade performance. Based on our empirical evidence, we find that current static reflection methods may lead to redundant, drift, and stubborn issues. To mitigate this, we introduce **I**nstruct-**o**f-**R**eflec**t**ion (**IoRT**), a novel and general reflection framework that leverages dynamic-meta instruction to enhance the iterative reflection capability of LLMs. Specifically, we propose the instructor driven by the meta-thoughts and self-consistency classifier, generates various instructions, including refresh, stop, and select, to guide the next reflection iteration. Our experiments demonstrate that IoRT achieves an average improvement of 10.1% over established baselines in mathematical and commonsense reasoning tasks, highlighting its efficacy and applicability. Our code is available at https://github.com/llp635/IoRT.
BibTeX
@inproceedings{liu2025instruct,
  author = {Liping Liu and Chunhong Zhang and Likang Wu and Chuang Zhao and Zheng Hu and Ming He and Jianping Fan},
  title = {{Instruct-of-Reflection: Enhancing Large Language Models Iterative Reflection Capabilities via Dynamic-Meta Instruction}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2025},
  url = {https://openreview.net/forum?id=sP6KmfYKfy},
}
Figure