Presented at EMNLP 2023, this research introduces a simple yet powerful framework for improving the reasoning capabilities of Large Language Models (LLMs) by dynamically choosing the best "reasoning recipe" for a given problem.
Introduction
The authors propose an automatic model selection method that uses an LLM to choose between two distinct reasoning strategies: natural language Chain-of-Thought (CoT) and programmatic Program-Aided Language Models (PAL). By leveraging the complementary strengths of these methods via in-context learning, the framework achieves new state-of-the-art results on benchmarks like GSM8K (96.8%) and SVAMP (93.7%) while significantly reducing computational costs compared to traditional ensemble methods.
The Reasoning Gap
While LLMs have become remarkably proficient at reasoning, their performance varies significantly across different task types. Currently, two dominant paradigms exist:
- Chain-of-Thought (CoT): The model explains its logic in natural language. This approach is flexible and excels at semantic understanding but is prone to "calculator errors" or arithmetic slips.
- Program-Aided Language Models (PAL): The model generates Python code to solve the problem. This ensures mathematical precision via an external interpreter but can fail if the model misinterprets the linguistic nuances of the question.
Most existing systems rely on a single method. The primary challenge addressed here is the lack of a dynamic, low-cost mechanism to decide which method is superior for a specific, unseen problem without requiring a specialized, trained "ranker" model.
Methodology: Automatic Model Selection
The authors introduce a two-stage Automatic Model Selection framework designed to bridge this gap:
- Solution Generation: For a given question, the LLM generates one CoT solution and one PAL solution.
- Model Selection: If the generated answers differ, the LLM acts as a "selector." It is prompted with the question and both candidate solutions in a multiple-choice format, tasked with identifying the correct one and providing a rationale.
Key Innovations
- In-Context Learning (ICL) for Selection: Instead of training a new model, the researchers use few-shot prompting to teach the LLM how to identify common errors in CoT (e.g., math slips) and PAL (e.g., logic errors in code).
- Theoretical Validation: The paper provides a mathematical proof showing that the success of this method depends on the diversity of the base models. Crucially, even a "weak" selector—one that isn't highly accurate—can still improve overall system performance if the underlying models fail in sufficiently different ways.
Datasets and Evaluation
The method was evaluated across eight reasoning benchmarks, primarily focusing on arithmetic reasoning (GSM8K, SVAMP, ASDIV, SingleOP, SingleEQ, AddSub, and MultiArith) and symbolic reasoning (Date Understanding from BIG-bench). The framework was tested using several backbone models, including Codex (code-davinci-002), ChatGPT (gpt-3.5-turbo), GPT-4, and open-source Llama 2 (7B and 13B).
Key Insights
Complementary Error Profiles
A case study of 100 error instances revealed that CoT and PAL have distinct failure modes. CoT excels at "Semantic Understanding" (grasping the intent), while PAL excels at "Computation Precision" and "Logical Decomposition."
The "Weak Selector" Advantage
The theoretical analysis suggests that a perfect judge is not required. A selector that is merely better than a random coin flip can provide significant gains, provided the models being judged have "dissimilar" error distributions.
Efficiency vs. Self-Consistency
Standard "Self-Consistency" methods (sampling a model 40-80 times) are computationally expensive. The authors found that their selection method with only 20 samples ("Ours@20") outperformed 80 samples of CoT alone, while using significantly fewer tokens.
Results
- State-of-the-Art Performance: The framework achieved 96.8% accuracy on GSM8K and 93.7% on SVAMP, surpassing previous benchmarks.
- Universal Improvement: Performance gains were consistent across all eight datasets and all backbone models. For instance, Llama 2 13B saw a +2.0% boost on GSM8K.
- Robustness: Even when the selector LLM differed from the generator LLM (e.g., using GPT-4 to select between ChatGPT's outputs), the performance gains remained significant.
Limitations and Future Work
While effective for math and symbolic logic, it remains unclear how this approach extends to purely creative or subjective tasks where code is not a viable alternative. Additionally, the authors noted an "option order bias" in LLMs, where models may prefer the first choice presented. Future work could involve selecting between a broader pool of methods, such as "ReAct" or "Least-to-Most" prompting, to further increase the diversity of candidate solutions.
Summary updated