Abstract
Recent work has shown how to prompt large language models with explanations to obtain strong performance on textual reasoning tasks, i.e., the chain-of-thought paradigm. However, subtly different explanations can yield widely varying downstream task accuracy. Explanations that have not been "tuned" for a task, such as off-the-shelf explanations written by non-experts, may lead to mediocre performance. This paper tackles the problem of how to optimize explanation-infused prompts in a blackbox fashion. We first generate sets of candidate explanations for each example in the prompt using a leave-one-out scheme, then find an effective combination of these explanations with a two-stage framework. We first evaluate explanations for each in-context example in isolation according to two proxy metrics, log likelihood and accuracy on new examples. Then, we search over combinations of explanations to find one that yields high performance against a silver-labeled development set. Across four textual reasoning tasks spanning question answering, mathematical reasoning, and natural language inference, results show that our proxy metrics correlate with ground truth accuracy and our overall method can effectively improve prompts over crowdworker annotations and naive search strategies.
Introduction: The Brittleness of Chain-of-Thought
Chain-of-Thought (CoT) prompting has revolutionized Large Language Model (LLM) performance on complex reasoning tasks by providing step-by-step examples. However, CoT is notoriously brittle: minor variations in how logic is explained can lead to significant swings in downstream accuracy. Currently, optimizing these prompts relies on manual "prompt engineering"—a subjective, time-consuming process that often yields sub-optimal results, particularly when using non-expert explanations. This research introduces a black-box framework to automatically select the most effective natural language explanations for a given task using unlabeled data.
Methodology: A Two-Stage Optimization Framework
The proposed framework identifies high-performing combinations of explanations through a systematic selection process:
- Candidate Generation: Using a "leave-one-out" approach, the LLM generates multiple alternative explanations for each example in a seed prompt. For instance, in an 8-shot prompt, 7 examples are used to prompt the model to generate a new explanation for the 8th.
- Proxy Metric Filtering: To avoid the exponential cost of testing every combination, the authors employ two "cheap" proxy metrics to rank candidates in isolation:
- One-shot Silver Accuracy ($S_{OSAcc}$): Measures how well a single explanation performs on an unlabeled development set.
- One-shot Log-Likelihood ($S_{OSLL}$): Measures the model's likelihood of generating correct answers for other prompt examples when conditioned on a specific explanation.
- Silver-Label Validation: An unlabeled dataset is transformed into a "silver-labeled" development set using majority voting across various prompts to establish likely correct answers.
- Final Selection: The framework identifies the top-N combinations via proxy metrics and validates them against the silver-labeled set to select the final prompt.
Experimental Results
The framework was evaluated across four diverse reasoning benchmarks: GSM (math), ECQA (commonsense QA), E-SNLI (natural language inference), and StrategyQA (multi-step reasoning).
The results demonstrate that the "OPTIMIZED" prompts consistently outperform human-written "SEED" prompts. On average, the method achieved a 4.3% improvement across all datasets. Notably, the gap between the worst and best random combinations of explanations reached as high as 20% on E-SNLI, highlighting the substantial performance gains left on the table by manual engineering.
Insights and Discussion
A key finding is that the content and quality of explanations provide more "headroom" for improvement than the order of examples. Furthermore, the success of one-shot proxies indicates that an explanation's individual performance is a strong predictor of its behavior in a few-shot context. The reliance on unlabeled data proves sufficient, as the "wisdom of the crowd" via majority voting provides a reliable enough signal for ranking candidates.
While effective, the method's computational cost remains a factor due to the LLM API calls required for generation and evaluation. Additionally, the quality of silver labels depends on the base model's inherent competence in the task domain. Future work may explore extending this framework to non-reasoning tasks like code generation or creative writing.
Summary updated