Prompt Optimization with Minimal Unlabeled Input via Meta-Reasoning

Yuran Sun, Chuan WuICML 2026 · OpenReview

Large Language Models (LLMs) are notoriously sensitive to how they are prompted. A minor tweak in phrasing or instruction order can drastically alter performance. While prompt optimization is critical, current methods face significant hurdles: supervised optimization requires expensive, often unavailable labeled data, while self-supervised methods struggle with "optimization ambiguity" (lacking a clear definition of a "good" output) or incur exorbitant computational costs.

To address this low-resource challenge, researchers have introduced the Meta-Reasoning Prompt Engineering Agent (MR.PEA). This self-supervised framework optimizes LLM prompts using just a task description and a single unlabeled example, achieving state-of-the-art results for pennies.

The Core Innovation: Meta-Reasoning

Instead of blindly guessing new prompts, MR.PEA acts as an autonomous agent that "reasons about reasoning." It iteratively builds a task-specific knowledge base, explicitly articulating the rules, strategies, and grading rubrics required for a given task. This shifts the optimization signal from relying on massive datasets to relying on structured, articulated knowledge.

MR.PEA operates through a five-phase cycle:

  1. Knowledge Curation: The agent derives "Abstract Strategies" (e.g., "break this math problem into steps") and "Evaluation Criteria" (e.g., "check for logical consistency"). Crucially, if the agent detects a gap in its domain knowledge, it adaptively performs a web search to supplement its understanding.
  2. Diverse Validation Generation: To prevent overfitting to the single provided example, MR.PEA generates a diverse pool of synthetic questions and reference answers.
  3. Strategy-Guided Refinement: Leveraging its curated strategies and historical feedback, the agent generates a new candidate prompt.
  4. Criteria-Based Pairwise Evaluation: The system tests the current best prompt against the new candidate using a validation example. It then "judges" the outputs based on its self-generated evaluation criteria.
  5. Dynamic Prompt Pool Management: A ranking system manages the prompts, utilizing an "Exploration Incentive" to reward novel, high-performing prompts and a decay factor to prevent older, lucky guesses from dominating the pool.

Key Insights and Performance

Evaluated on the GSM8K (grade-school math) and Big-Bench Hard (BBH) datasets, MR.PEA demonstrated remarkable efficiency and effectiveness.

  • Significant Performance Gains: MR.PEA achieved an average performance boost of 7.4% over the best existing self-supervised baseline (SPO). On BBH math tasks specifically, it delivered a staggering 18.7% improvement.
  • Unmatched Cost-Effectiveness: By minimizing API calls and focusing on reasoning rather than brute-force iteration, optimization costs plummeted to just **$0.01 per task**—drastically cheaper than competitors like GLaPE ($0.11 per task).
  • Retrieval-Augmented Strategy: Unlike "closed-book" optimizers, MR.PEA uses web search to discover strategies, not just facts. For instance, on a task requiring humorous name editing, it searched for linguistic humor techniques to refine its prompt.
  • Cross-Model Generalization: Prompts optimized by MR.PEA on one model successfully transferred to others, including Llama-3 and DeepSeek-V3, often allowing smaller models to match or exceed the performance of larger models using standard Chain-of-Thought prompting.

Limitations and Future Directions

While highly effective, MR.PEA has a few constraints. Its reliance on external knowledge means that poor web search results could mislead the optimization process. Furthermore, the method requires a base model with a strong foundational ability to follow complex instructions; it may struggle with smaller, non-instruction-tuned models. Finally, operating without ground-truth labels introduces a risk of bias. The model could theoretically reinforce its own internal biases or hallucinate logically flawed evaluation criteria, highlighting the need for future work to audit these self-supervised loops.

Ultimately, MR.PEA proves that with the right meta-reasoning framework, LLMs can teach themselves to be better problem solvers using minimal data and minimal compute.

Summary updated

Abstract

Prompt optimization is critical for maximizing the performance of large language models (LLMs). However, it often relies on costly labeled data. Self-supervised methods reduce data dependency, but they suffer from optimization ambiguity or high computational costs. To address these limitations, we propose the Meta-Reasoning Prompt Engineering Agent (MR.PEA), a self-supervised prompt optimization framework that operates with minimal input. MR.PEA leverages meta-reasoning to iteratively build task-specific knowledge, including problem-solving strategies and evaluation criteria, while adaptively retrieving external information to enhance its understanding. This knowledge guides the generation of diverse validation examples, targeted prompt refinement, and comprehensive quality assessments. Experiments on GSM8K and Big-Bench Hard show that MR.PEA outperforms existing baselines, achieving an average performance gain of 7.4% with an optimization cost as low as $0.01 per task.
BibTeX
@inproceedings{sun2026prompt,
  author = {Yuran Sun and Chuan Wu},
  title = {{Prompt Optimization with Minimal Unlabeled Input via Meta-Reasoning}},
  booktitle = {International Conference on Machine Learning},
  year = {2026},
  url = {https://openreview.net/forum?id=RdbSgk61V0},
}
Figure