TL;DR
This paper introduces ContextFocus, a novel activation steering technique that significantly improves the ability of Large Language Models (LLMs) to adhere to provided context, especially when that context conflicts with the model's internal knowledge. It achieves this without requiring model fine-tuning, offering an efficient and scalable solution for more reliable LLM outputs in real-world applications.
What Problem Does This Paper Address?
Large Language Models (LLMs) are trained on vast amounts of data, embedding a significant amount of "parametric knowledge." While this is powerful, the world is constantly changing, and new information emerges. When LLMs are used in applications that rely on up-to-date external information (like retrieval-augmented generation or RAG systems), they often struggle to prioritize this new context. Instead, they tend to default to their older, memorized parametric knowledge, leading to "unfaithful" outputs that contradict the provided evidence. This is particularly problematic in knowledge-conflict scenarios, where the external context directly clashes with the LLM's internal knowledge. The paper aims to solve this by making LLMs more faithful to external context.
Methods
The core innovation is ContextFocus, a lightweight activation steering approach. Activation steering works by identifying specific directions in the LLM's internal activation space that correspond to a desired behavior. In this case, the desired behavior is "contextual faithfulness."
Here's how it works:
- Vector Generation: They create a "steering vector" by contrasting the LLM's internal activations when processing prompts that either rely on context or default to parametric knowledge. This contrast helps isolate the direction that pushes the model towards context-adherence. They found that jointly contrasting both contextual evidence and system instructions (rather than just one or the other) yields the most effective steering vector.
- Activation Injection: During inference (when the LLM is generating a response), this steering vector is added to the model's activations at a specific layer. This "nudges" the model's internal representations, guiding its generation process to favor the provided context.
- Key Features:
- No Fine-tuning: ContextFocus operates on frozen LLMs, meaning it doesn't require retraining the entire model, making it much more efficient and scalable.
- Lightweight: It only requires a single forward pass during inference, adding minimal computational overhead.
- Complementary: It can be combined with prompting strategies for even better results.
Datasets
- ConFiQA: This is the primary benchmark used for evaluation. It's designed to specifically test contextual faithfulness in knowledge-conflict settings and includes three subsets: QA (single-hop QA), MR (multi-hop reasoning with a single counterfactual), and MC (multi-hop reasoning with multiple counterfactuals).
- NQ-SWAP: Used for vector generation and some ablation studies. This dataset contains questions where retrieved context contradicts the LLM's prior knowledge.
Key Insights and Ideas
- Joint Contrast is Key: The most significant finding is that the most effective steering vectors are created by contrasting both contextual evidence and system instructions against a baseline that uses neither. Isolating just context or just system instructions leads to less improvement. This suggests that the interplay between explicit guidance and factual evidence is crucial for robust context adherence.
- Data Efficiency: ContextFocus requires surprisingly little data (as few as 1.5k examples) to generate a highly effective steering vector, which quickly converges to a stable direction. This is a major advantage over fine-tuning methods that require much larger datasets.
- Layer Matters: The effectiveness of activation steering is highly dependent on the layer at which the vector is injected. The paper systematically identifies optimal layers for different models, showing that intermediate layers often yield the best results.
- Scalability: The method works effectively on larger models (like Llama-3.1-70B), demonstrating its potential for real-world deployment.
Key Results
- Significant Faithfulness Improvement: ContextFocus consistently improves context faithfulness (measured by the
psmetric) across various benchmarks and models, often outperforming strong baselines like ContextDPO and COIECD. For example, on the ConFiQA benchmark, ContextFocus achieved significantly higherpsvalues compared to unsteered models and other methods. - Reduced Reliance on Parametric Knowledge: The method effectively reduces the model's tendency to rely on outdated or incorrect parametric knowledge, as reflected by lower
poandMRmetrics. - Efficiency: ContextFocus offers comparable inference speed to unsteered generation and is significantly faster than decoding-based methods like COIECD, which require multiple passes. It also avoids the substantial training costs associated with fine-tuning methods.
- Robustness: The method shows robustness across different model sizes and maintains effectiveness even when combined with prompting strategies.
Limitations and Future Work
- Focus on Specific Tasks: The current evaluation is primarily focused on question-answering under knowledge-conflict settings. The authors acknowledge that extending this to more open-ended or long-form generation tasks, where faithfulness errors are more diffuse, is an important direction for future work.
- Fixed Multiplier: The current approach uses a fixed steering multiplier (
m=2) across all inputs. While this provides a stable operating point, the authors suggest that adaptive or input-conditional control of steering strength could further improve performance by tailoring the intervention to specific inputs, potentially benefiting more examples without degrading fluency.
Summary updated