ContextFocus: Activation Steering for Contextual Faithfulness in Large Language Models

Nikhil Anand, Shwetha Somasundaram, Anirudh Phukan, Apoorv Saxena, Koyel MukherjeearXiv 2026 · arXiv · OpenReview

TL;DR

This paper introduces ContextFocus, a novel activation steering technique that significantly improves the ability of Large Language Models (LLMs) to adhere to provided context, especially when that context conflicts with the model's internal knowledge. It achieves this without requiring model fine-tuning, offering an efficient and scalable solution for more reliable LLM outputs in real-world applications.

What Problem Does This Paper Address?

Large Language Models (LLMs) are trained on vast amounts of data, embedding a significant amount of "parametric knowledge." While this is powerful, the world is constantly changing, and new information emerges. When LLMs are used in applications that rely on up-to-date external information (like retrieval-augmented generation or RAG systems), they often struggle to prioritize this new context. Instead, they tend to default to their older, memorized parametric knowledge, leading to "unfaithful" outputs that contradict the provided evidence. This is particularly problematic in knowledge-conflict scenarios, where the external context directly clashes with the LLM's internal knowledge. The paper aims to solve this by making LLMs more faithful to external context.

Methods

The core innovation is ContextFocus, a lightweight activation steering approach. Activation steering works by identifying specific directions in the LLM's internal activation space that correspond to a desired behavior. In this case, the desired behavior is "contextual faithfulness."

Here's how it works:

  1. Vector Generation: They create a "steering vector" by contrasting the LLM's internal activations when processing prompts that either rely on context or default to parametric knowledge. This contrast helps isolate the direction that pushes the model towards context-adherence. They found that jointly contrasting both contextual evidence and system instructions (rather than just one or the other) yields the most effective steering vector.
  2. Activation Injection: During inference (when the LLM is generating a response), this steering vector is added to the model's activations at a specific layer. This "nudges" the model's internal representations, guiding its generation process to favor the provided context.
  3. Key Features:
    • No Fine-tuning: ContextFocus operates on frozen LLMs, meaning it doesn't require retraining the entire model, making it much more efficient and scalable.
  4. Lightweight: It only requires a single forward pass during inference, adding minimal computational overhead.
  5. Complementary: It can be combined with prompting strategies for even better results.

Datasets

  • ConFiQA: This is the primary benchmark used for evaluation. It's designed to specifically test contextual faithfulness in knowledge-conflict settings and includes three subsets: QA (single-hop QA), MR (multi-hop reasoning with a single counterfactual), and MC (multi-hop reasoning with multiple counterfactuals).
  • NQ-SWAP: Used for vector generation and some ablation studies. This dataset contains questions where retrieved context contradicts the LLM's prior knowledge.

Key Insights and Ideas

  • Joint Contrast is Key: The most significant finding is that the most effective steering vectors are created by contrasting both contextual evidence and system instructions against a baseline that uses neither. Isolating just context or just system instructions leads to less improvement. This suggests that the interplay between explicit guidance and factual evidence is crucial for robust context adherence.
  • Data Efficiency: ContextFocus requires surprisingly little data (as few as 1.5k examples) to generate a highly effective steering vector, which quickly converges to a stable direction. This is a major advantage over fine-tuning methods that require much larger datasets.
  • Layer Matters: The effectiveness of activation steering is highly dependent on the layer at which the vector is injected. The paper systematically identifies optimal layers for different models, showing that intermediate layers often yield the best results.
  • Scalability: The method works effectively on larger models (like Llama-3.1-70B), demonstrating its potential for real-world deployment.

Key Results

  • Significant Faithfulness Improvement: ContextFocus consistently improves context faithfulness (measured by the ps metric) across various benchmarks and models, often outperforming strong baselines like ContextDPO and COIECD. For example, on the ConFiQA benchmark, ContextFocus achieved significantly higher ps values compared to unsteered models and other methods.
  • Reduced Reliance on Parametric Knowledge: The method effectively reduces the model's tendency to rely on outdated or incorrect parametric knowledge, as reflected by lower po and MR metrics.
  • Efficiency: ContextFocus offers comparable inference speed to unsteered generation and is significantly faster than decoding-based methods like COIECD, which require multiple passes. It also avoids the substantial training costs associated with fine-tuning methods.
  • Robustness: The method shows robustness across different model sizes and maintains effectiveness even when combined with prompting strategies.

Limitations and Future Work

  • Focus on Specific Tasks: The current evaluation is primarily focused on question-answering under knowledge-conflict settings. The authors acknowledge that extending this to more open-ended or long-form generation tasks, where faithfulness errors are more diffuse, is an important direction for future work.
  • Fixed Multiplier: The current approach uses a fixed steering multiplier (m=2) across all inputs. While this provides a stable operating point, the authors suggest that adaptive or input-conditional control of steering strength could further improve performance by tailoring the intervention to specific inputs, potentially benefiting more examples without degrading fluency.

Summary updated

Abstract

Large Language Models (LLMs) encode vast amounts of parametric knowledge during pre-training. As world knowledge evolves, effective deployment increasingly depends on their ability to faithfully follow externally retrieved context. When such evidence conflicts with the model's internal knowledge, LLMs often default to memorized facts, producing unfaithful outputs. In this work, we introduce ContextFocus, a lightweight activation steering approach that improves context faithfulness in such knowledge-conflict settings while preserving fluency and efficiency. Unlike prior approaches, our solution requires no model finetuning and incurs minimal inference-time overhead, making it highly efficient. We evaluate ContextFocus on the ConFiQA benchmark, comparing it against strong baselines including ContextDPO, COIECD, and prompting-based methods. Furthermore, we show that our method is complementary to prompting strategies and remains effective on larger models. Extensive experiments show that ContextFocus significantly improves contextual-faithfulness. Our results highlight the effectiveness, robustness, and efficiency of ContextFocus in improving contextual-faithfulness of LLM outputs.
BibTeX
@misc{anand2026contextfocus,
  author = {Nikhil Anand and Shwetha Somasundaram and Anirudh Phukan and Apoorv Saxena and Koyel Mukherjee},
  title = {{ContextFocus: Activation Steering for Contextual Faithfulness in Large Language Models}},
  year = {2026},
  eprint = {2601.04131},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2601.04131},
}
Figure