TL;DR
Self-consistency is a simple, unsupervised decoding strategy that significantly boosts the reasoning capabilities of large language models (LLMs). By replacing traditional greedy decoding with a "sample-and-marginalize" approach—sampling multiple diverse reasoning paths and taking a majority vote on the final answer—it achieves striking performance gains across arithmetic, symbolic, and commonsense reasoning benchmarks.
What Problem Does This Paper Address?
While large language models excel at many natural language tasks, multi-step reasoning (like solving math word problems or answering complex commonsense questions) remains a major bottleneck. Chain-of-thought (CoT) prompting helps by forcing the model to generate intermediate reasoning steps before delivering a final answer.
However, CoT prompting traditionally relies on greedy decoding, where the model selects the single most likely token at each step. This approach suffers from several flaws:
- It lacks exploration and can get stuck in locally optimal but globally incorrect reasoning paths.
- It is highly vulnerable to minor errors early in the generation sequence, which cascade and ruin the final answer.
- It fails to leverage a core human intuition: there are often multiple completely different ways to think through a problem and arrive at the correct conclusion.
Methods
The authors introduce self-consistency, an entirely unsupervised, off-the-shelf decoding strategy that acts as a "self-ensemble" over a single pre-trained model.
Instead of generating just one greedy reasoning path, the method follows three steps:
- Prompt: Prompt the LLM using standard chain-of-thought exemplars.
- Sample: Sample a diverse set of candidate reasoning paths from the model's decoder using standard sampling techniques (e.g., temperature sampling, top-$k$, or nucleus sampling).
- Marginalize: Parse the final answers from all generated paths and aggregate them using a simple majority vote (unweighted sum).
The core idea is that while incorrect reasoning paths can lead to a wide variety of random wrong answers, correct reasoning paths will consistently converge on the exact same correct answer.
Datasets
The authors evaluated self-consistency across a broad suite of reasoning benchmarks:
- Arithmetic Reasoning: GSM8K (grade-school math), SVAMP, AQuA-RAT, MultiArith, AddSub, and ASDiv.
- Commonsense Reasoning: CommonsenseQA, StrategyQA, and the AI2 Reasoning Challenge (ARC-Easy and ARC-Challenge).
- Symbolic Reasoning: Last letter concatenation and Coinflip tasks.
- General NLP Tasks (where CoT sometimes hurts performance): BoolQ, HotpotQA, e-SNLI, ANLI, and RTE.
Key Insights and Ideas
- Agreement Implies Correctness: Diverse correct reasoning processes tend to yield the same final answer, whereas flawed reasoning processes diverge into many different incorrect answers.
- Unweighted Majority Vote is Sufficient: Interestingly, weighting the votes by the model's generation probability doesn't significantly outperform a simple, unweighted majority vote. This is because the model tends to view different sampled paths as "similarly likely."
- Built-in Uncertainty Estimation: The degree of consistency (the percentage of sampled paths that agree on the final answer) is highly correlated with the accuracy of the output. This allows self-consistency to serve as a reliable confidence metric, effectively helping the model "know when it doesn't know."
Key Results
Self-consistency was tested across four major language models: UL2-20B, GPT-3 (175B), LaMDA-137B, and PaLM-540B. It consistently delivered massive performance boosts:
- New State-of-the-Art: Combined with PaLM-540B or GPT-3, it achieved unprecedented accuracy on arithmetic tasks, including a +17.9% absolute gain on GSM8K and +12.2% on AQuA.
- Robust to Scale and Sampling: The performance gains hold true across different model sizes and are highly robust to various sampling parameters (like temperature adjustments).
- Diminishing Returns with Sample Size: While sampling more paths consistently improves accuracy, the gains begin to saturate around 10 to 20 paths, meaning users can capture most of the benefits without excessive computation.
Limitations and Future Work
- Computational Cost: Because it requires generating multiple outputs (typically 40 paths in the paper's main experiments) for a single query, self-consistency incurs significantly higher inference and computational costs.
- Fixed Answer Constraints: The strategy currently relies on tasks where the final answer can be easily parsed into a fixed set (e.g., numbers or multiple-choice letters). Extending this to open-ended text generation remains an open challenge, as it requires defining a robust metric for "consistency" between free-form text responses.
- Factuality of Rationales: While self-consistency fixes the final answer, the model can still occasionally output flawed or nonsensical intermediate reasoning steps that happen to stumble upon the correct answer.
Summary updated