TL;DR
This paper introduces a novel approach to enhance the in-context learning (ICL) capabilities of large language models (LLMs) by leveraging sparse autoencoders (SAEs). They propose two key innovations: "Feature Detection through Prompt Variation" (FDPV) for efficiently identifying relevant SAE features, and "Selective In-Context Steering" (SISTER) for applying these features to improve ICL performance, particularly in classification tasks. This work demonstrates a significant, robust improvement in ICL performance, marking a step towards more interpretable and controllable LLMs.
What Problem Does This Paper Address?
Large Language Models (LLMs) have shown remarkable "in-context learning" (ICL) abilities, meaning they can perform new tasks by simply being shown a few examples within the prompt, without any parameter updates. However, a major challenge in controlling and improving this capability is understanding how LLMs achieve it and how to steer them towards desired behaviors. Mechanistic interpretability tools like Sparse Autoencoders (SAEs) have emerged as powerful ways to decompose LLM representations into interpretable features. While SAEs have been used for probing and steering, their application to improving general ICL performance has been limited. Existing methods for identifying useful SAE features are computationally expensive, and steering methods are often not tailored to the nuances of ICL, leading to sensitivity to hyperparameters and limited generalizability. This paper aims to bridge this gap by proposing efficient methods for both identifying and applying SAE features to boost ICL performance.
Methods
The paper introduces two main methodological contributions:
- Feature Detection through Prompt Variation (FDPV): This is a novel, unsupervised, and computationally efficient method for identifying SAE features that are useful for steering. The core idea is to compare SAE activation patterns between a "baseline prompt" and a "prompt-variant" that is known to perform better. Features that are consistently more (or less) active in the better-performing prompt are considered strong candidates for steering. This method avoids the costly feature ablation techniques used previously.
- Selective In-Context Steering (SISTER): This is a new steering method specifically designed for ICL classification tasks. It's inspired by recent findings that LLMs use "label words" (e.g., "Positive," "Negative") in few-shot examples as anchors for classification. SISTER focuses the intervention of SAE steering vectors only on these label tokens, rather than on all tokens in the prompt as in standard approaches. This targeted intervention aims for more effective performance boosts and greater robustness to hyperparameters.
Datasets
The authors evaluated their approach on four widely used ICL classification datasets:
- SemEval-14 Laptops and Restaurants (ABSC): For aspect-based sentiment classification.
- AGNews: For news topic classification.
- EmoC: For emotion classification from short dialogues.
- ARC-Challenge: For multiple-choice question answering (used to test generalization beyond classification).
They used publicly available SAEs trained on Gemma2 and Llama3 models.
Key Insights and Ideas
- SAEs are surprisingly sensitive to subtle prompt differences: Even when a "prompt-variant" only repeats the test query without adding new semantic content, SAEs can detect and highlight significant activation differences in their features. This suggests SAEs are powerful tools for capturing nuanced LLM behavior.
- Label words are crucial anchors in ICL classification: The paper reinforces the idea that LLMs rely heavily on specific words (like class labels) to perform ICL tasks. This insight is the foundation for SISTER's targeted steering approach.
- Targeted steering is more robust and effective: By focusing interventions on specific "anchor" tokens (label words) rather than indiscriminately on all tokens, SISTER achieves better performance gains and is significantly less sensitive to the steering strength hyperparameter ($\alpha$). This creates a "wider Goldilocks zone" for effective steering.
- FDPV effectively identifies task-relevant features: The analysis shows that features identified by FDPV as being more active in the better-performing prompt (FDPV+) generally have higher task-specific scores (nPMI), suggesting FDPV successfully isolates features that contribute positively to performance.
Key Results
- Consistent performance improvements: The proposed FDPV + SISTER method consistently outperforms baseline ICL and standard steering approaches across all tested models (Gemma2-9B-IT, Gemma2-2B-IT, Llama3-8B-IT) and datasets.
- Significant gains: For example, on Gemma2-9B-IT, FDPV + SISTER achieved an average performance improvement of 3.5% across tasks.
- Robustness: SISTER demonstrates significantly greater robustness to the steering hyperparameter $\alpha$ compared to standard steering methods, maintaining performance over a much wider range of $\alpha$ values.
- Generalizability: SISTER also shows effectiveness in multiple-choice question answering tasks, though with smaller gains, suggesting its applicability beyond pure classification.
Limitations and Future Work
- SAE accessibility: A primary constraint is the reliance on sparse autoencoders (SAEs), which are computationally expensive to train. This limits the evaluation to models for which pre-trained SAEs are available (Gemma2 and Llama3).
- Layer-wise analysis: Due to resource limitations, comprehensive layer-wise analyses were not conducted, which could offer further insights.
Future work could involve extending this approach to other model architectures and conducting deeper investigations across different layers of LLMs.
Summary updated