Supportiveness-based Knowledge Rewriting for Retrieval-augmented Language Modeling

Zile Qiao, Wei Ye, Yong Jiang, Tong Mo, Pengjun Xie, Weiping Li, Fei Huang, Shikun Zhang

Abstract

Retrieval-augmented language models (RALMs) have recently shown great potential in mitigating the limitations of implicit knowledge in LLMs, such as untimely updating of the latest expertise and unreliable retention of long-tail knowledge. However, since the external knowledge base, as well as the retriever, can not guarantee reliability, potentially leading to the knowledge retrieved not being helpful or even misleading for LLM generation. In this paper, we introduce Supportiveness-based Knowledge Rewriting (SKR), a robust and pluggable knowledge rewriter inherently optimized for LLM generation. Specifically, we introduce the novel concept of "supportiveness"—which represents how effectively a knowledge piece facilitates downstream tasks—by considering the perplexity impact of augmented knowledge on the response text of a white-box LLM. Based on knowledge supportiveness, we first design a training data curation strategy for our rewriter model, effectively identifying and filtering out poor or irrelevant rewrites to improve data efficacy. We then introduce the direct preference optimization (DPO) algorithm to align the generated rewrites to optimal supportiveness, guiding the rewriter model to summarize augmented content that better improves the final response. Comprehensive evaluations across six popular knowledge-intensive tasks and four LLMs have demonstrated the effectiveness and superiority of SKR. With only 7B parameters, SKR has shown better knowledge rewriting capability over GPT-4, the current state-of-the-art general-purpose LLM.

Introduction

Retrieval-Augmented Generation (RAG) is the standard approach for providing Large Language Models (LLMs) with access to real-time or specialized data. However, retrievers frequently introduce "noise"—irrelevant or misleading text chunks—that can degrade model performance.

A significant hurdle in training rewriters to filter this noise is Internal Knowledge Bias. If an LLM already possesses the knowledge required to answer a query, its confidence (measured by perplexity) remains relatively stable even when provided with high-quality retrieved evidence. Previous training methods that relied solely on raw perplexity changes failed to distinguish between poor retrieved information and cases where the model was already an expert. This leads to weak training signals for the rewriter, as it cannot accurately gauge the utility of the provided context.

Methodology: Supportiveness-based Knowledge Rewriting (SKR)

To address these challenges, the authors propose Supportiveness-based Knowledge Rewriting (SKR), a two-pronged approach to align a rewriter model with the specific needs of a downstream LLM.

The Supportiveness Metric

Instead of measuring absolute perplexity reduction, SKR introduces a "supportiveness score." This metric compares the model's confidence with and without retrieved information, normalized to account for the model's baseline internal knowledge. This ensures the rewriter focuses on providing information the LLM actually lacks.

Training Pipeline

  1. Supportiveness-based Rewrite Data Generation (SRDG): GPT-4 generates multiple candidate rewrites for a large-scale dataset. A "white-box" LLM then scores these candidates using the supportiveness metric, categorizing them as successful, failed, or irrelevant.
  2. Supportiveness-guided Alignment: After initial Supervised Fine-Tuning (SFT) on successful rewrites, the model undergoes Direct Preference Optimization (DPO). This explicitly teaches the rewriter to prefer high-supportiveness rewrites over lower-scoring ones.
The SKR framework: Supportiveness-based Rewrite Data Generation and Supportiveness-guided Alignment.

The final model, SKR-7B, is built on the Mistral-7B architecture and is designed to act as a specialized "bridge" that provides exactly what the downstream LLM is missing.

Experimental Results

SKR was evaluated on six knowledge-intensive benchmarks, including MMLU, Natural Questions (NQ), and TriviaQA (TQA), using Llama-2 (7B and 70B) as the backbone models.

Performance and Compression

SKR-7B consistently outperformed state-of-the-art general-purpose models like GPT-4 in the rewriting task. On the Llama-2-7B model, SKR improved average Exact Match (EM) scores by 10.01 points compared to raw retrieval. Beyond accuracy, SKR achieved over 7x compression of input text, significantly reducing computational overhead and latency while simultaneously improving response quality.

Performance comparison of SKR against various rewriters and retrieval baselines across multiple benchmarks.

Robustness and Irrelevance Detection

A key feature of SKR is its ability to handle cases where retrieved information is entirely unhelpful. In such scenarios, the model learns to output "irrelevant," signaling the LLM to rely on its internal knowledge. This makes the system far more robust to "bad" or "random" retrieval than competing methods like RECOMP or FILCO.

Conclusion

SKR demonstrates that a specialized 7B parameter model, when aligned via supportiveness, can surpass the performance of much larger general-purpose models in RAG pipelines. By accounting for internal knowledge bias, SKR provides a more efficient and accurate way to bridge the gap between external knowledge bases and LLM generation. Future work may explore dynamic rewriters that adapt as LLMs grow more knowledgeable, requiring even more precise filtering of redundant information.

Summary updated

Abstract

Retrieval-augmented language models (RALMs) have recently shown great potential in mitigating the limitations of implicit knowledge in LLMs, such as untimely updating of the latest expertise and unreliable retention of long-tail knowledge. However, since the external knowledge base, as well as the retriever, can not guarantee reliability, potentially leading to the knowledge retrieved not being helpful or even misleading for LLM generation. In this paper, we introduce Supportiveness-based Knowledge Rewriting (SKR), a robust and pluggable knowledge rewriter inherently optimized for LLM generation. Specifically, we introduce the novel concept of "supportiveness"--which represents how effectively a knowledge piece facilitates downstream tasks--by considering the perplexity impact of augmented knowledge on the response text of a white-box LLM. Based on knowledge supportiveness, we first design a training data curation strategy for our rewriter model, effectively identifying and filtering out poor or irrelevant rewrites (e.g., with low supportiveness scores) to improve data efficacy. We then introduce the direct preference optimization (DPO) algorithm to align the generated rewrites to optimal supportiveness, guiding the rewriter model to summarize augmented content that better improves the final response. Comprehensive evaluations across six popular knowledge-intensive tasks and four LLMs have demonstrated the effectiveness and superiority of SKR. With only 7B parameters, SKR has shown better knowledge rewriting capability over GPT-4, the current state-of-the-art general-purpose LLM.
BibTeX
@inproceedings{qiao2025supportiveness,
  author = {Zile Qiao and Wei Ye and Yong Jiang and Tong Mo and Pengjun Xie and Weiping Li and Fei Huang and Shikun Zhang},
  title = {{Supportiveness-based Knowledge Rewriting for Retrieval-augmented Language Modeling}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2025},
  url = {https://openreview.net/forum?id=uLy476ydll},
}
Figure