On Meta-Prompting

Adrian de Wynter, Xun Wang, Qilong Gu, Si-Qing Chen

Abstract: Modern large language models (LLMs) are capable of interpreting input strings as instructions, or prompts, and carry out tasks based on them. Unlike traditional learners, LLMs cannot use back-propagation to obtain feedback, and condition their output in situ in a phenomenon known as in-context learning (ICL). Many approaches to prompting and pre-training these models involve the automated generation of these prompts, also known as meta-prompting, or prompting to obtain prompts. However, they do not formally describe the properties and behavior of the LLMs themselves. We propose a theoretical framework based on category theory to generalize and describe ICL and LLM behavior when interacting with users. Our framework allows us to obtain formal results around task agnosticity and equivalence of various meta-prompting approaches. Using our framework and experimental results we argue that meta-prompting is more effective than basic prompting at generating desirable outputs.

Introduction

While Large Language Models (LLMs) have revolutionized AI through In-Context Learning (ICL), our understanding of prompting remains largely empirical. LLMs are incredibly sensitive to prompt wording, leading to the rise of "prompt engineering," which is often a trial-and-error process. This paper addresses the lack of a rigorous theoretical characterization of how LLMs adapt to tasks or interact with users. By introducing a formal mathematical framework based on Category Theory, the authors explain how LLMs interpret instructions and prove that "meta-prompting"—using an LLM to generate its own task-specific prompts—is a task-agnostic strategy that outperforms traditional, fixed system prompts.

Theoretical Framework

The researchers turn to Category Theory to focus on the relationships (morphisms) between objects rather than internal mechanics. They define a category called Prompt, where objects are sets of strings (tokens) and morphisms are instructions that transform one set into another.

Commutative diagram representing the relationship between input context, generated prompts, and final outputs.

In this framework, specific applications (like summarization or translation) are modeled as subcategories. Meta-prompting is modeled as an internal hom ($Z^X$), representing the set of all possible maps from context $X$ to output $Z$. A meta-prompt is a morphism that takes a user's context and selects the "best" specific instruction from this set. This abstraction allows the authors to ignore the "black box" mechanics of neural networks and prove that meta-prompting is "system prompt-agnostic."

A 2-morphism representing the transformation or equivalence between different prompting strategies.

Furthermore, the paper establishes Structural Equivalence, suggesting that many different automated prompting techniques are essentially doing the same thing mathematically. This explains why diverse approaches often yield similarly improved results: they are all mechanisms to choose the most relevant instruction based on specific input rather than relying on fixed bottlenecks like standard system prompts.

Experimental Validation

To validate their theory, the authors conducted experiments on two distinct tasks using models evaluated by GPT-4 and ranked by professional human annotators:

  1. Ideation: Suggesting ways to improve text (sampled from WikiText-103 and TOEFL11).
  2. Creativity: Story continuation or text insertion (sampled from DailyMail/CNN and TOEFL11).

Key Results

  • Preference for Meta-Prompts: In the Creativity task, meta-generated prompts were ranked in the top 3 selections 71% of the time, significantly beating hardcoded baselines.
  • Better Downstream Outputs: Meta-generated outputs were in the top 3 61% of the time, showing that better prompts lead to better final text.
  • Statistical Significance: Improvements were statistically significant ($p < 0.01$) under a Wilcoxon signed-rank test for both tasks.
  • Qualitative Edge: Annotators found meta-generated prompts to be more "contextualized," whereas baseline prompts were often too generic for complex writing tasks.

Limitations and Future Work

The current framework assumes the LLM is deterministic for simplicity. In reality, LLMs are stochastic, and the authors suggest using Markov categories in the future to account for probability. Additionally, the framework currently treats prompts as total functions, though they are technically partial functions (e.g., "summarize" does not apply to a single word). Future directions include modeling complex behaviors like Chain-of-Thought prompting and linking the framework more directly to human preference data to refine the current "coarse-grained" model.

Summary updated

Abstract

Modern large language models (LLMs) are capable of interpreting input strings as instructions, or prompts, and carry out tasks based on them. Unlike traditional learners, LLMs cannot use back-propagation to obtain feedback, and condition their output in situ in a phenomenon known as in-context learning (ICL). Many approaches to prompting and pre-training these models involve the automated generation of these prompts, also known as meta-prompting, or prompting to obtain prompts. However, they do not formally describe the properties and behavior of the LLMs themselves. We propose a theoretical framework based on category theory to generalize and describe ICL and LLM behavior when interacting with users. Our framework allows us to obtain formal results around task agnosticity and equivalence of various meta-prompting approaches. Using our framework and experimental results we argue that meta-prompting is more effective than basic prompting at generating desirable outputs.
BibTeX
@misc{wynter2023meta,
  author = {Adrian de Wynter and Xun Wang and Qilong Gu and Si-Qing Chen},
  title = {{On Meta-Prompting}},
  year = {2023},
  eprint = {2312.06562},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2312.06562},
}
Figure