TL;DR
This paper introduces SPRINT, a novel framework that uses contrastive learning to identify and prune specific attention heads in large language models (LLMs) that, surprisingly, improve reasoning capabilities. By dynamically selecting which heads to prune based on the question, SPRINT enhances LLM performance on challenging math reasoning tasks without increasing inference time.
What Problem Does This Paper Address?
Large Language Models (LLMs) have shown remarkable progress in understanding and generating human language, leading to applications in content creation, virtual assistance, and problem-solving. However, a significant challenge remains their reliability and accuracy, especially in tasks requiring complex reasoning like mathematical problem-solving or automated theorem proving. Current methods to improve LLM reasoning often involve iterative generation strategies (e.g., best-of-N sampling) or search techniques, which can be computationally expensive. Model pruning, typically used for efficiency, is generally assumed to degrade performance. This paper investigates whether pruning specific components of LLMs, specifically attention heads, can improve reasoning abilities, contrary to conventional wisdom.
Methods
The core of this work is the SPRINT (Structured PRunIng for diverSe reasoNing opTimization) framework. SPRINT leverages a surprising empirical observation: pruning certain attention heads can actually boost an LLM's reasoning performance.
- Contrastive Learning for Head Embeddings: SPRINT first learns embeddings for each attention head-layer pair. This is done using a contrastive learning approach where the goal is to align the embedding of a question with the embeddings of heads whose pruning leads to correct answers for that question. A novel "Diversity-Promoted Contrastive Loss" is employed.
- Loss Function: The loss function has two main components:
- Alignment Term: Encourages the question embedding to be closer to the embeddings of heads that, when pruned, result in correct answers. This is achieved using a softmax formulation over negative squared Euclidean distances.
- Diversity Term: Acts as a regularizer to ensure that head embeddings are distinct, especially for heads that tend to yield similar outcomes. This promotes a richer, more discriminative representation of heads.
- Inference: At inference time, for a given question, SPRINT identifies the optimal head to prune by finding the head whose learned embedding is closest to the question's embedding.
- Best-of-N Integration: SPRINT integrates with existing best-of-N reasoning strategies. Instead of sampling from the base model, it samples from a set of models where different heads have been pruned, guided by the learned embeddings. This allows for generating diverse, high-performing candidate answers without additional computational overhead during inference.
Datasets
The paper uses the following datasets for evaluation:
- MATH500: A subset of the MATH dataset, consisting of 500 samples for evaluation.
- GSM8K: A dataset of 8th-grade math problems.
The authors mention subsampling 1,500 training examples from the respective training sets for training the attention head embeddings.
Key Insights and Ideas
- Pruning Can Improve Reasoning: The most significant insight is that selectively pruning certain attention heads can improve an LLM's reasoning performance, rather than degrade it. This challenges the conventional view of pruning solely for efficiency.
- Redundancy and Noise: Some attention heads might introduce redundant or noisy signals into the model's decision-making process, leading to incorrect reasoning. Pruning these heads can act as a form of regularization, enhancing accuracy.
- Head-Question Specificity: The impact of pruning is highly dependent on both the specific head and the input question. This suggests a need for dynamic, question-aware pruning strategies.
- SPRINT's Effectiveness: The proposed SPRINT framework successfully leverages this phenomenon by learning to identify which heads to prune for specific questions, leading to significant performance gains.
Key Results
- Outperforms Baselines: SPRINT consistently outperforms traditional best-of-N sampling and random head selection strategies on the MATH500 and GSM8K datasets.
- Accuracy Gains: The method achieves higher accuracy on mathematical reasoning tasks. For instance, pruning head 0 in layer 15 of Qwen2.5-Math-1.5B-Instruct increased accuracy by 16% on the "Counting & Probability" category.
- No Inference-Time Overhead: A key advantage is that SPRINT does not introduce additional computational cost during inference, as the pruning decision is made based on pre-learned embeddings.
- Specialized Models Benefit More: Specialized mathematical models showed higher median gains and more frequent large improvements compared to general-purpose models.
Limitations and Future Work
- Limited Model Architectures/Sizes: The study was limited to a specific range of model architectures and sizes due to space constraints.
- Mathematical Domain Focus: The experiments primarily focused on the mathematical domain. The authors suggest extending the approach to other scientific domains requiring LLM reasoning.
- Catastrophic Pruning Case: The paper did not explore cases where pruning a single head might lead to a catastrophic performance drop.
- Future Directions: The authors suggest extending the work to other scientific domains and exploring the implications of catastrophic pruning.
Summary updated