Slim-SC: Thought Pruning for Efficient Scaling with Self-Consistency

Colin Hong, Xu Guo, Anand Chaanan Singh, Esha Choukse, Dmitrii UstiugovEMNLP 2025

TL;DR

This paper introduces Slim-SC, a novel method for making Self-Consistency (SC), a powerful but computationally expensive technique for improving LLM reasoning, more efficient. By pruning redundant reasoning chains based on their semantic similarity at the "thought" level, Slim-SC significantly reduces latency and resource usage without sacrificing accuracy, offering a practical alternative to standard SC.

What Problem Does This Paper Address?

Large Language Models (LLMs) have shown impressive reasoning capabilities, often enhanced by techniques like Chain-of-Thought (CoT) prompting. However, a more advanced method called Self-Consistency (SC) further boosts performance by generating multiple reasoning chains in parallel and using majority voting to select the final answer. While effective, SC incurs a substantial computational overhead, limiting its widespread adoption. Existing attempts to speed up SC often involve heuristics or model-based confidence scores, which can be unreliable or require additional training. This paper addresses the inefficiency of SC by theoretically and empirically analyzing its bottlenecks and proposing a new pruning strategy that targets redundant reasoning paths.

Methods

The core of Slim-SC is a thought-level pruning strategy. Instead of generating all N reasoning chains and then voting, Slim-SC intelligently prunes chains during their generation.

Here's how it works:

  • Pruning Delay: To avoid prematurely removing chains that might develop unique reasoning paths, pruning is delayed until chains have progressed beyond an initial "warm-up" phase (a set number of generation steps).
  • Similarity Check: After the delay, at regular intervals, the system embeds the latest "thought" (a segment of the reasoning chain) from each active chain. These embeddings are then compared to identify highly similar thoughts between different chains.
  • Pruning: If a thought from chain ci is found to be highly similar (similarity > Tsim, typically 0.98) to a thought from chain cj, one of the chains is selected for pruning.
  • Random Pruning (RP): A simple baseline where one of the similar chains is chosen randomly to be pruned.
  • Diversity-based Pruning (DP): A more sophisticated approach that prunes the chain exhibiting less internal diversity among its own thoughts, aiming to preserve more unique reasoning paths.
  • Algorithm: The process continues until only one or a few chains remain, or until all chains complete. The final answer is then derived from the remaining, non-pruned chains.

Datasets

The paper evaluates Slim-SC on three benchmarks:

  • GPQA Diamond: A graduate-level dataset for complex STEM reasoning.
  • AIME-2024: A dataset for high-difficulty math problems.
  • AQUA-RAT: A dataset for algebraic word problems requiring multi-step numerical reasoning.

The LLMs used for evaluation are R1-Distill-Qwen-14B and QwQ-32B.

Key Insights and Ideas

  • Redundancy is the Enemy: The core insight is that many reasoning chains, especially incorrect ones, become highly similar in their intermediate thoughts. This redundancy is a major source of inefficiency in standard SC.
  • Correct vs. Incorrect Chains Cluster: Empirically, correct and incorrect reasoning chains form distinct clusters in the semantic space. This separation allows for targeted pruning of redundant incorrect chains without significantly harming the diversity of correct ones.
  • Longer Chains Often Mean Incorrect: The study found that chains leading to incorrect answers tend to be longer, contributing to the "wait-for-all" bottleneck in SC. Pruning these longer, redundant chains directly addresses this latency issue.
  • Pruning Preserves Diversity: Unlike methods that might prune based on confidence scores or simply reduce the number of chains, Slim-SC's similarity-based approach aims to preserve a diverse set of unique reasoning paths, which is crucial for robust decision-making.
  • Pruning Delay is Crucial: Applying pruning too early can be detrimental, as initial thoughts might be superficially similar but diverge later. A warm-up delay allows chains to develop their distinct reasoning strategies.

Key Results

  • Accuracy Maintained or Improved: Slim-SC (both RP and DP variants) consistently achieves accuracy comparable to or better than standard SC across all datasets and models. For example, Slim-SC (RP) improved accuracy on GPQA Diamond by +0.5 pp over SC.
  • Significant Latency Reduction: Slim-SC (DP) reduces inference latency by up to 45% (on GPQA) and 43% (on AIME) compared to standard SC.
  • Reduced Token Usage and KV Cache: Slim-SC reduces the number of generated tokens by up to 32% (on AIME) and mean KV Cache usage by up to 10pp compared to SC.
  • Efficiency Advantage over ESC: While Early-Stopping Self-Consistency (ESC) also aims for efficiency, Slim-SC offers a much better latency profile (ESC is over 2.9x slower on GPQA) and a more balanced accuracy-latency trade-off.
  • Robustness: Slim-SC is robust to the choice of pruning strategy (RP vs. DP) and performs well across a range of similarity thresholds and pruning delays.

Limitations and Future Work

  • Metric Design: The authors acknowledge that while similarity patterns are effective, a more sophisticated metric could potentially bridge the gap between oracle accuracy and actual accuracy.
  • Pruning vs. Reuse: The current method prunes similar chains. Future work could explore merging partial reasoning segments or KV-cache states to improve both accuracy and computational reuse.
  • Scalability: The evaluation was limited to specific LLMs and math reasoning tasks. Extending the evaluation to larger models and more diverse datasets is suggested to assess generalizability.

Summary updated

BibTeX
@inproceedings{hong2025slim,
  author = {Colin Hong and Xu Guo and Anand Chaanan Singh and Esha Choukse and Dmitrii Ustiugov},
  title = {{Slim-SC: Thought Pruning for Efficient Scaling with Self-Consistency}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2025},
}
Figure