TL;DR
This paper introduces PROPEL, a novel framework for optimizing prompts for small and medium-sized Large Language Models (LLMs). By leveraging "expert priors" – structured knowledge about LLM strengths and weaknesses – PROPEL significantly improves prompt effectiveness, leading to better performance on various NLP tasks compared to existing methods. This is particularly crucial for resource-constrained LLMs that struggle with complex instructions.
What Problem Does This Paper Address?
Large Language Models (LLMs) have revolutionized NLP, but their performance is highly sensitive to prompt quality. Crafting effective prompts, known as prompt engineering, is a resource-intensive process often requiring domain expertise and trial-and-error. This challenge is amplified for smaller and medium-sized LLMs (1-8 billion parameters) because they generally exhibit weaker instruction-following capabilities compared to their larger counterparts. Existing automated prompt optimization methods often focus on larger models or lack the structured knowledge needed to guide the optimization process effectively for these smaller models. This paper aims to bridge this gap by developing a method that specifically enhances prompt optimization for resource-efficient LLMs.
Methods
PROPEL employs a novel iterative optimization framework that orchestrates three distinct LLM roles:
- Responder LLM: Generates responses to a given prompt and input context.
- Judge LLM: Evaluates the generated responses based on predefined criteria, providing feedback and a score. This role is crucial for providing detailed, qualitative insights beyond simple numerical scores.
- Optimizer LLM: Refines the prompt based on the feedback and scores from the Judge LLM, crucially guided by "Expert Priors."
Expert Priors are a key innovation. They are derived from a one-time evaluation of the Responder LLM on a comprehensive set of prompt principles (50 in total). These principles are classified as "easy" (emphasis rules) or "hard" (avoidance rules) for the specific LLM. The Optimizer LLM uses these rules to steer prompt generation, reinforcing good practices and avoiding problematic ones. The framework also incorporates a "State" that stores historical prompts, feedback, and scores, providing contextual learning for the Optimizer.
Datasets
The paper utilizes a combination of public and in-house datasets:
- QMSum: A public dataset for query-based summarization of meeting transcripts.
- MT Bench: A public benchmark for evaluating multi-turn chat capabilities.
- In-house dataset: A proprietary dataset of contact center conversations used for query-based summarization and entity extraction tasks. This dataset was carefully curated and annotated by domain experts.
Key Insights and Ideas
- Expert Priors are Crucial for Smaller LLMs: The most significant insight is that structured knowledge about LLM capabilities (expert priors) is vital for effective prompt optimization, especially for smaller models that struggle with complex instructions.
- The "Self" Approach is Insufficient: Using the same small/medium LLM as both Responder and Judge/Optimizer (PROPEL-Self) performs significantly worse than using a more powerful external LLM (like GPT-4) for the Judge and Optimizer roles. This highlights the need for a capable evaluator and optimizer.
- History Matters: Maintaining a history of prompts, feedback, and scores (State History) significantly improves prompt optimization effectiveness, demonstrating the value of contextual learning in iterative processes.
- Feedback + Scores are Better than Either Alone: Combining feedback and scores in the State provides complementary insights, leading to more effective prompt refinements than using either signal in isolation.
- "Hard" Prompts Benefit Most from Optimization: While "easy" prompts see improvements, "hard" prompts (those that are more challenging for the LLM) show a greater relative improvement after PROPEL optimization, indicating its power in overcoming inherent model limitations.
Key Results
PROPEL consistently outperforms baselines across various tasks and models.
- Summarization: Improvements of 10-24% over initial prompts, and at least 12% over baselines.
- Entity Extraction: Improvements of 5-16% over initial prompts, and at least 9% over baselines.
- Llama-3.2 1B: PROPEL improves response quality by 21% with expert priors.
- Llama-3.2 3B: PROPEL improves response quality by 9% with expert priors.
- Impact of Priors: "Good Priors" led to the highest improvement (10-15%), significantly outperforming "No Priors," "Random Priors," and even "Bad Priors" (which showed a performance decline).
Limitations and Future Work
The authors acknowledge several limitations:
- Applicability of Expert Priors: The one-time evaluation to derive expert priors might not be suitable for highly dynamic requirements.
- Reliance on Reference Responses: The framework relies on reference responses for evaluation, which can be a bottleneck for tasks lacking readily available references.
- Task Diversity: The current evaluation is limited to summarization, entity extraction, and QnA. Broader testing on more complex tasks is needed.
- Model Scale: The study is confined to models up to 8B parameters. Extending to larger models would demonstrate PROPEL's adaptability.
- Language: Experiments were conducted solely on English tasks, leaving multilingual capabilities for future exploration.
Future work could involve exploring more dynamic methods for deriving expert priors, developing strategies for tasks without reference responses, and extending PROPEL to larger models and multilingual settings.
Summary updated