Abstract
Retrieval-augmented language models (RALMs) have recently shown great potential in mitigating the limitations of implicit knowledge in LLMs, such as untimely updating of the latest expertise and unreliable retention of long-tail knowledge. However, since the external knowledge base, as well as the retriever, can not guarantee reliability, potentially leading to the knowledge retrieved not being helpful or even misleading for LLM generation. In this paper, we introduce Supportiveness-based Knowledge Rewriting (SKR), a robust and pluggable knowledge rewriter inherently optimized for LLM generation. Specifically, we introduce the novel concept of "supportiveness"—which represents how effectively a knowledge piece facilitates downstream tasks—by considering the perplexity impact of augmented knowledge on the response text of a white-box LLM. Based on knowledge supportiveness, we first design a training data curation strategy for our rewriter model, effectively identifying and filtering out poor or irrelevant rewrites to improve data efficacy. We then introduce the direct preference optimization (DPO) algorithm to align the generated rewrites to optimal supportiveness, guiding the rewriter model to summarize augmented content that better improves the final response. Comprehensive evaluations across six popular knowledge-intensive tasks and four LLMs have demonstrated the effectiveness and superiority of SKR. With only 7B parameters, SKR has shown better knowledge rewriting capability over GPT-4, the current state-of-the-art general-purpose LLM.
Introduction
Retrieval-Augmented Generation (RAG) is the standard approach for providing Large Language Models (LLMs) with access to real-time or specialized data. However, retrievers frequently introduce "noise"—irrelevant or misleading text chunks—that can degrade model performance.
A significant hurdle in training rewriters to filter this noise is Internal Knowledge Bias. If an LLM already possesses the knowledge required to answer a query, its confidence (measured by perplexity) remains relatively stable even when provided with high-quality retrieved evidence. Previous training methods that relied solely on raw perplexity changes failed to distinguish between poor retrieved information and cases where the model was already an expert. This leads to weak training signals for the rewriter, as it cannot accurately gauge the utility of the provided context.
Methodology: Supportiveness-based Knowledge Rewriting (SKR)
To address these challenges, the authors propose Supportiveness-based Knowledge Rewriting (SKR), a two-pronged approach to align a rewriter model with the specific needs of a downstream LLM.
The Supportiveness Metric
Instead of measuring absolute perplexity reduction, SKR introduces a "supportiveness score." This metric compares the model's confidence with and without retrieved information, normalized to account for the model's baseline internal knowledge. This ensures the rewriter focuses on providing information the LLM actually lacks.
Training Pipeline
- Supportiveness-based Rewrite Data Generation (SRDG): GPT-4 generates multiple candidate rewrites for a large-scale dataset. A "white-box" LLM then scores these candidates using the supportiveness metric, categorizing them as successful, failed, or irrelevant.
- Supportiveness-guided Alignment: After initial Supervised Fine-Tuning (SFT) on successful rewrites, the model undergoes Direct Preference Optimization (DPO). This explicitly teaches the rewriter to prefer high-supportiveness rewrites over lower-scoring ones.
The final model, SKR-7B, is built on the Mistral-7B architecture and is designed to act as a specialized "bridge" that provides exactly what the downstream LLM is missing.
Experimental Results
SKR was evaluated on six knowledge-intensive benchmarks, including MMLU, Natural Questions (NQ), and TriviaQA (TQA), using Llama-2 (7B and 70B) as the backbone models.
Performance and Compression
SKR-7B consistently outperformed state-of-the-art general-purpose models like GPT-4 in the rewriting task. On the Llama-2-7B model, SKR improved average Exact Match (EM) scores by 10.01 points compared to raw retrieval. Beyond accuracy, SKR achieved over 7x compression of input text, significantly reducing computational overhead and latency while simultaneously improving response quality.
Robustness and Irrelevance Detection
A key feature of SKR is its ability to handle cases where retrieved information is entirely unhelpful. In such scenarios, the model learns to output "irrelevant," signaling the LLM to rely on its internal knowledge. This makes the system far more robust to "bad" or "random" retrieval than competing methods like RECOMP or FILCO.
Conclusion
SKR demonstrates that a specialized 7B parameter model, when aligned via supportiveness, can surpass the performance of much larger general-purpose models in RAG pipelines. By accounting for internal knowledge bias, SKR provides a more efficient and accurate way to bridge the gap between external knowledge bases and LLM generation. Future work may explore dynamic rewriters that adapt as LLMs grow more knowledgeable, requiring even more precise filtering of redundant information.
Summary updated