Self-Consistency Improves Chain of Thought Reasoning in Language Models

Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V. Le, Ed H. Chi, Denny Zhou

TL;DR

This paper introduces "self-consistency," a novel decoding strategy for large language models that significantly improves their reasoning abilities. Instead of relying on a single, greedily generated reasoning path, self-consistency samples multiple diverse paths and selects the most consistent final answer, leading to substantial accuracy gains on complex reasoning tasks.

What Problem Does This Paper Address?

Large language models (LLMs) have shown impressive capabilities, but their reasoning abilities have been a persistent challenge. While "chain-of-thought" (CoT) prompting, which encourages LLMs to generate step-by-step reasoning, has improved performance, it still relies on a single, greedily chosen reasoning path. This can lead to errors due to the model's inherent limitations and local optima in its generated thought process. The authors aim to overcome this by leveraging the idea that complex problems can often be solved through multiple valid reasoning pathways, and that agreement across these pathways can indicate a more reliable answer.

Methods

The core innovation is the self-consistency decoding strategy. It replaces the standard greedy decoding used in CoT prompting with a "sample-and-marginalize" approach:

  1. Prompting: The LLM is prompted using chain-of-thought examples.
  2. Sampling: Instead of picking the single most likely reasoning path, the model samples a diverse set of reasoning paths. This is achieved using standard sampling techniques like temperature sampling, top-k, or nucleus sampling.
  3. Marginalization: The sampled reasoning paths might lead to different final answers. Self-consistency aggregates these answers by choosing the one that appears most frequently (majority vote) across the sampled paths. This leverages the intuition that correct reasoning paths, even if diverse, will converge on the same correct answer, while incorrect paths are less likely to do so.

This method is unsupervised, works off-the-shelf with pre-trained models, and doesn't require additional training or human annotations. It's described as a "self-ensemble" for a single model.

Datasets

The authors evaluated self-consistency on a variety of reasoning benchmarks:

  • Arithmetic Reasoning: GSM8K, SVAMP, AQuA-RAT, MultiArith, ASDiv.
  • Commonsense Reasoning: CommonsenseQA, StrategyQA, ARC-challenge, ARC-easy.
  • Symbolic Reasoning: Last letter concatenation, Coinflip.
  • Common NLP Tasks: BoolQ, HotpotQA, e-SNLI, ANLI, RTE.

They used publicly available datasets and did not introduce new benchmarks.

Key Insights and Ideas

  • Diversity is Key: The core idea is that multiple reasoning paths can lead to the same correct answer, and by sampling these diverse paths, we can increase confidence in the final answer.
  • "Self-Ensemble": Self-consistency acts like an ensemble method but operates on a single model, making it more accessible and efficient than traditional multi-model ensembles.
  • Robustness to Imperfect Prompts: The method shows resilience to minor errors or variations in the few-shot prompts, which is crucial for practical applications where prompt engineering can be challenging.
  • Uncertainty Estimation: The degree of consistency among sampled answers can serve as an indicator of the model's confidence, allowing it to "know when it doesn't know."

Key Results

Self-consistency significantly boosted performance across all tested LLMs (UL2-20B, GPT-3-175B, LaMDA-137B, PaLM-540B) and reasoning tasks.

  • Arithmetic Reasoning: Achieved new state-of-the-art results, with gains up to +17.9% absolute accuracy on GSM8K (PaLM-540B), +12.2% on AQuA, and +11.0% on SVAMP.
  • Commonsense Reasoning: Significant improvements were observed, such as +6.4% on StrategyQA and +3.9% on ARC-challenge.
  • Robustness: The method showed consistent gains even when chain-of-thought prompting alone might hurt performance on some NLP tasks.
  • Comparison to Other Methods: Self-consistency outperformed sample-and-rank, beam search, and ensemble-based approaches in terms of accuracy gains.

Limitations and Future Work

  • Computational Cost: Self-consistency incurs higher computational costs due to sampling multiple paths. However, the authors note that performance often saturates quickly with a small number of sampled paths (e.g., 5-10), mitigating this issue.
  • Non-Natural Language Reasoning: While self-consistency works for equation-based reasoning, the gains are smaller due to less diversity in shorter reasoning paths.
  • Rationale Quality: LLMs can sometimes generate nonsensical or factually incorrect reasoning paths, even if they lead to the correct final answer. Future work could focus on grounding these rationales better and improving their factuality and safety.
  • Data Generation: The authors suggest using self-consistency to generate better supervised data for fine-tuning models to achieve accurate predictions in a single inference run.

Summary updated

Abstract

Chain-of-thought prompting combined with pre-trained large language models has achieved encouraging results on complex reasoning tasks. In this paper, we propose a new decoding strategy, self-consistency, to replace the naive greedy decoding used in chain-of-thought prompting. It first samples a diverse set of reasoning paths instead of only taking the greedy one, and then selects the most consistent answer by marginalizing out the sampled reasoning paths. Self-consistency leverages the intuition that a complex reasoning problem typically admits multiple different ways of thinking leading to its unique correct answer. Our extensive empirical evaluation shows that self-consistency boosts the performance of chain-of-thought prompting with a striking margin on a range of popular arithmetic and commonsense reasoning benchmarks, including GSM8K (+17.9%), SVAMP (+11.0%), AQuA (+12.2%), StrategyQA (+6.4%) and ARC-challenge (+3.9%).
BibTeX
@inproceedings{wang2023self,
  author = {Xuezhi Wang and Jason Wei and Dale Schuurmans and Quoc V. Le and Ed H. Chi and Denny Zhou},
  title = {{Self-Consistency Improves Chain of Thought Reasoning in Language Models}},
  booktitle = {International Conference on Learning Representations},
  year = {2023},
  url = {https://openreview.net/forum?id=1PL1NIMMrw},
}
Figure