Accommodate Knowledge Conflicts in Retrieval-augmented LLMs: Towards Reliable Response Generation in the Wild

Jiatai Wang, Zhiwei Xu, Di Jin, Xuewen Yang, Tao LiarXiv 2025 · DOI · arXiv

Introduction

Retrieval-Augmented Generation (RAG) has become a cornerstone for reducing hallucinations in Large Language Models (LLMs) by providing them with external, up-to-date information. However, this approach introduces a significant challenge: knowledge conflicts. These occur when the retrieved external context contradicts the model’s internal parametric memory—the knowledge it acquired during pre-training. Such conflicts often lead to high uncertainty and unreliable responses.

Researchers from Nankai University have addressed this issue with Swin-VIB, a novel framework grounded in information theory. Swin-VIB acts as a "plug-and-play" adapter that helps LLMs navigate these contradictions by filtering out ambiguous information and focusing on high-signal context.

The Theoretical Landscape of Uncertainty

The core of the problem lies in how LLMs prioritize conflicting sources of information. The researchers found that model uncertainty is not linear. Instead, it follows a "rise-then-fall" pattern based on the "information difference" between internal memory and external context.

The relationship between model uncertainty (\psi) and the information difference (Ic - Is). Uncertainty peaks in the 'Confused' region where the conflict is most ambiguous.

As shown in the figure above, when the context is either purely supplementary or purely conflicting, the model remains relatively confident. However, when the information is a mix of both, the model enters a state of confusion, leading to peak uncertainty. Swin-VIB is designed to identify and discard the "undecidable" information in this middle region.

Methodology: The Swin-VIB Framework

To operationalize this theoretical insight, the authors proposed the Sliding-window Variational Information Bottleneck (Swin-VIB). The framework consists of several key components:

  1. Sliding Window Segmentation: The retrieved context is divided into small, fixed-length windows (optimally 7 tokens). This allows the model to evaluate the information density and conflict level at a granular level.
  2. Variational Information Bottleneck (VIB): Small VIB modules are attached to the LLM's layers. These modules are trained to compress input representations while retaining only the features necessary to predict the information difference.
  3. Adaptive Filtering: Each window is assigned a probability. Windows containing clear conflicting or supplementary information (high information difference) are "accepted," while noisy or ambiguous windows are "rejected."
A comparison between Naive RAG and the Swin-VIB approach. Swin-VIB uses a sliding window to filter the context, reducing uncertainty and leading to a correct, definitive response.

By filtering the context before it reaches the final generation stage, Swin-VIB guides the LLM toward a more confident and accurate answer, effectively resolving the "tug-of-war" between internal and external knowledge.

Experimental Evaluation

The researchers evaluated Swin-VIB across several benchmarks, including ConflictQA, DRUID, and TruthfulQA. The framework was tested on various state-of-the-art LLMs such as Llama 2, Qwen, and DeepSeek.

Key Results

  • Accuracy and Robustness: Swin-VIB consistently outperformed existing baselines like CD² and CK-PLUG. In open-ended QA tasks on TruthfulQA, the integration of Swin-VIB improved Exact Match (EM) scores by over 11.14%.
  • Correction vs. Resistance: The model achieved an optimal balance between correcting wrong internal beliefs (Correction Rate) and resisting misleading external information (Resistance Rate).
  • Efficiency: The computational overhead is minimal. For a Llama2-7B model, Swin-VIB adds only approximately 0.39 seconds of latency, making it a practical solution for real-world RAG pipelines.

Conclusion and Future Directions

Swin-VIB represents a significant step toward more reliable RAG systems. By using information theory to filter out the "mushy middle" of undecidable information, it allows LLMs to maintain high confidence even in the face of conflicting data.

While highly effective, future work could improve cross-segment coherence for very complex documents and expand the framework to multilingual settings and more diverse tasks beyond question answering. Nonetheless, Swin-VIB provides a principled, "plug-and-play" solution for one of the most persistent problems in modern LLM deployment.

Summary updated

Abstract

The proliferation of large language models (LLMs) has significantly advanced intelligent systems. Unfortunately, LLMs often face knowledge conflicts between internal memory and retrieved external information, arising from misinformation, biases, or outdated knowledge. These conflicts undermine response reliability and introduce uncertainty in decision-making. In this work, we analyze how LLMs navigate knowledge conflicts from an information-theoretic perspective and reveal that when conflicting and supplementary information exhibit significant differences, LLMs confidently resolve their preferences and alleviate the uncertainty during their response generation. When this difference is ambiguous, LLMs experience considerable uncertainty about their generation. Based on this insight, we propose Swin-VIB, a novel framework that integrates a pipeline of variational information bottleneck models to adapt the retrieved information difference, facilitating robust response generation of LLMs even in conflicting contexts. Extensive experiments confirm our theoretical analysis and demonstrate the performance of Swin-VIB. Notably, Swin-VIB outperforms all competitive baselines in terms of the accuracy of the multiple-choice task, while improving the EM values in the open-ended QA task by at least 11.14%.
BibTeX
@misc{wang2025accommodate,
  author = {Jiatai Wang and Zhiwei Xu and Di Jin and Xuewen Yang and Tao Li},
  title = {{Accommodate Knowledge Conflicts in Retrieval-augmented LLMs: Towards Reliable Response Generation in the Wild}},
  year = {2025},
  eprint = {2504.12982},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2504.12982},
}
Figure