Enhancing Large Language Models' Situated Faithfulness to External Contexts

Yukun Huang, Sanxing Chen, Hongyi Cai, Bhuwan Dhingra

Abstract

Large Language Models (LLMs) are often augmented with external information as contexts, but this external information can sometimes be inaccurate or even intentionally misleading. We argue that robust LLMs should demonstrate situated faithfulness, dynamically calibrating their trust in external information based on their confidence in the internal knowledge and the external context. To benchmark this capability, we evaluate LLMs across several QA datasets, including a newly created dataset called RedditQA featuring in-the-wild incorrect contexts sourced from Reddit posts. We show that when provided with both correct and incorrect contexts, both open-source and proprietary models tend to overly rely on external information, regardless of its factual accuracy. To enhance situated faithfulness, we propose two approaches: Self-Guided Confidence Reasoning (SCR) and Rule-Based Confidence Reasoning (RCR). SCR enables models to self-access the confidence of external information relative to their own internal knowledge to produce the most accurate answer. RCR, in contrast, extracts explicit confidence signals from the LLM and determines the final answer using predefined rules. Our results show that for LLMs with strong reasoning capabilities, such as GPT-4o and GPT-4o mini, SCR outperforms RCR, achieving improvements of up to 24.2% over a direct input augmentation baseline. Conversely, for a smaller model like Llama-3-8B, RCR outperforms SCR. Fine-tuning SCR with our proposed Confidence Reasoning Direct Preference Optimization (CR-DPO) method improves performance on both seen and unseen datasets, yielding an average improvement of 8.9% on Llama-3-8B.

Introduction: The Challenge of Blind Faithfulness

Retrieval-Augmented Generation (RAG) is a standard technique used to ground Large Language Models (LLMs) in external data, thereby reducing hallucinations and providing up-to-date information. However, this reliance creates a vulnerability: "blind faithfulness." Current LLMs often exhibit an over-reliance on provided context, even when that context is factually incorrect or intentionally misleading.

We propose that a truly robust model should exhibit situated faithfulness. This capability allows a model to dynamically calibrate its trust, leaning on its internal knowledge when it is certain and the context is dubious, while remaining open to external information when its own knowledge is lacking or the evidence is compelling.

Methodology: SCR, RCR, and CR-DPO

To address knowledge conflicts, we evaluate two primary reasoning frameworks designed to foster situated faithfulness:

  1. Self-Guided Confidence Reasoning (SCR): This approach prompts the model to use its internal reasoning to resolve conflicts. In Implicit SCR, the model is simply warned that the context might be unreliable. In Explicit SCR, the model uses Chain-of-Thought (CoT) to explicitly verbalize its internal confidence and evaluate the context's reliability before deciding on a final answer.
  2. Rule-Based Confidence Reasoning (RCR): This method extracts explicit confidence signals (such as numerical scores) for both the model's internal knowledge and the external context. Predefined rules are then applied to select the final answer (e.g., "if internal confidence is high and context confidence is low, ignore the context").
Concept illustration of SCR and RCR workflows for resolving knowledge conflicts.

Furthermore, we introduce Confidence Reasoning Direct Preference Optimization (CR-DPO). This novel fine-tuning method trains models to prefer "correct" reasoning paths—those that successfully identify deceptive contexts—over paths where the model was misled.

Benchmarking with RedditQA

We evaluate these methods across several standard QA benchmarks (FreshQA, NaturalQA, TriviaQA, PopQA, and ClashEval) and a newly introduced dataset: RedditQA. RedditQA contains "in-the-wild" incorrect contexts sourced from real Reddit posts. This provides a more realistic evaluation of how models handle human-written misinformation compared to synthetic perturbations.

Results: Reasoning Capability and Deception Detection

Our results indicate that the effectiveness of these methods depends heavily on the model's underlying reasoning capability. Stronger models like GPT-4o excel at SCR, achieving improvements of up to 24.2% over direct input augmentation. Smaller models, such as Llama-3-8B, initially struggle with the complex reasoning required for SCR and perform better with the structured rules of RCR.

However, fine-tuning with CR-DPO significantly boosts the performance of smaller models. For Llama-3-8B, CR-DPO yielded an average improvement of 8.9% across datasets. A critical advantage of CR-DPO is its ability to help models identify "red herrings"—misleading details designed to trick the model into accepting a false context.

Qualitative comparison showing CR-DPO successfully identifying a misleading context (red herring) where ExplicitSCR failed.

Conclusion

Situated faithfulness is a vital capability for the next generation of RAG-enabled LLMs. While high-capacity models can often reason through conflicts using SCR, smaller models benefit significantly from structured rule-based approaches or targeted fine-tuning via CR-DPO. Future research will focus on extending these methods to long-form generation and multi-document conflict resolution.

Summary updated

Abstract

Large Language Models (LLMs) are often augmented with external information as contexts, but this external information can sometimes be inaccurate or even intentionally misleading. We argue that robust LLMs should demonstrate situated faithfulness, dynamically calibrating their trust in external information based on their confidence in the internal knowledge and the external context. To benchmark this capability, we evaluate LLMs across several QA datasets, including a newly created dataset called RedditQA featuring in-the-wild incorrect contexts sourced from Reddit posts. We show that when provided with both correct and incorrect contexts, both open-source and proprietary models tend to overly rely on external information, regardless of its factual accuracy. To enhance situated faithfulness, we propose two approaches: Self-Guided Confidence Reasoning (SCR) and Rule-Based Confidence Reasoning (RCR). SCR enables models to self-access the confidence of external information relative to their own internal knowledge to produce the most accurate answer. RCR, in contrast, extracts explicit confidence signals from the LLM and determines the final answer using predefined rules. Our results show that for LLMs with strong reasoning capabilities, such as GPT-4o and GPT-4o mini, SCR outperforms RCR, achieving improvements of up to 24.2% over a direct input augmentation baseline. Conversely, for a smaller model like Llama-3-8B, RCR outperforms SCR. Fine-tuning SCR with our proposed Confidence Reasoning Direct Preference Optimization (CR-DPO) method improves performance on both seen and unseen datasets, yielding an average improvement of 8.9% on Llama-3-8B. In addition to quantitative results, we offer insights into the relative strengths of SCR and RCR. Our findings highlight promising avenues for improving situated faithfulness in LLMs. The data and code are released.
BibTeX
@inproceedings{huang2025enhancing,
  author = {Yukun Huang and Sanxing Chen and Hongyi Cai and Bhuwan Dhingra},
  title = {{Enhancing Large Language Models' Situated Faithfulness to External Contexts}},
  booktitle = {International Conference on Learning Representations},
  year = {2025},
  url = {https://openreview.net/forum?id=K2jOacHUlO},
}
Figure