Enhancing Noise Robustness of Retrieval-Augmented Language Models with Adaptive Adversarial Training

Feiteng Fang, Yuelin Bai, Shiwen Ni, Min Yang, Xiaojun Chen, Ruifeng Xu

Abstract

Large Language Models (LLMs) exhibit substantial capabilities yet encounter challenges, including hallucination, outdated knowledge, and untraceable reasoning processes. Retrieval-augmented generation (RAG) has emerged as a promising solution, integrating knowledge from external databases to mitigate these challenges. However, inappropriate retrieved passages can potentially hinder the LLMs' capacity to generate comprehensive and high-quality responses. Prior RAG studies on the robustness of retrieval noises often confine themselves to a limited set of noise types, deviating from real-world retrieval environments and limiting practical applicability. In this study, we initially investigate retrieval noises and categorize them into three distinct types, reflecting real-world environments. We analyze the impact of these various retrieval noises on the robustness of LLMs. Subsequently, we propose a novel RAG approach known as Retrieval-augmented Adaptive Adversarial Training (RAAT). RAAT leverages adaptive adversarial training to dynamically adjust the model's training process in response to retrieval noises. Concurrently, it employs multi-task learning to ensure the model's capacity to internally recognize noisy contexts. Extensive experiments demonstrate that the LLaMA-2 7B model trained using RAAT exhibits significant improvements in F1 and EM scores under diverse noise conditions.

Introduction

Retrieval-Augmented Generation (RAG) has become the standard architecture for grounding Large Language Models (LLMs) in factual, up-to-date information. By fetching relevant documents from an external database, RAG systems can significantly reduce hallucinations. However, there is a critical catch: retrievers are far from perfect. When a retriever returns "noise"—irrelevant or misleading text—the LLM often gets distracted, leading to incorrect answers.

In this study, researchers introduce RAAT (Retrieval-augmented Adaptive Adversarial Training), a framework designed to make LLMs "noise-aware" and significantly more robust to the messy reality of document retrieval.

The Challenge of Retrieval Noise

While RAG is powerful, it introduces a "noise robustness" problem. Most existing research assumes the retriever is mostly accurate, but in real-world settings, the retrieved context often contains three distinct types of noise:

  1. Irrelevant Noise: Text completely unrelated to the query.
  2. Relevant Noise: Text that looks related (shares keywords) but doesn't actually contain the answer.
  3. Counterfactual Noise: Text that is topically relevant but contains factually incorrect or misleading information.

Current LLMs are surprisingly sensitive to these distractions. If a model is trained only on "clean" data, it learns to over-rely on the provided context, making it vulnerable when that context is wrong. The authors argue that models need to dynamically adapt to varying levels and types of noise during the generation process.

The RAAT Framework

RAAT is a novel training framework that enhances robustness through three core components:

The RAAT training framework integrates adaptive adversarial training with a noise-awareness classification task to improve LLM robustness.
  • Adaptive Adversarial Training: Instead of training on a static mix of clean and noisy data, RAAT uses a "min-max" optimization strategy. For every query, the model calculates the loss for four scenarios: the "golden" context and the three types of noise. It then prioritizes updating the model parameters based on the scenario that produced the highest loss (the "hardest" example).
  • Regularization ($L_{reg}$): To prevent the model from overfitting to one specific type of noise, the authors introduce a regularization term that minimizes the variance between the maximum and minimum losses across the different noise types.
  • Incorporating Noise Awareness ($L_{cls}$): This multi-task learning approach adds a small linear layer beneath the LLM to perform an auxiliary classification task: identifying which of the four context types it is currently reading. This forces the model to develop internal representations that can distinguish between high-quality and low-quality information.

Experimental Results

The researchers established a new benchmark called RAG-Bench to evaluate their approach, built upon three widely used open-domain QA datasets: Natural Questions (NQ), TriviaQA, and WebQ. Testing on the LLaMA-2 7B model revealed significant gains:

  • Superior Performance: RAAT achieved an average F1 score of 86.35 and an Exact Match (EM) score of 82.23 across all noise conditions.
  • Significant Gains: Compared to the standard LLaMA-2 7B baseline, RAAT improved the average F1 score by nearly 30 points (from 56.68 to 86.35).
  • Outperforming Specialized Baselines: RAAT consistently beat RALM-multiple (a model trained on multiple noise types without the adaptive/classification components) by about 2% in F1, demonstrating that the way you train on noise matters as much as the noise itself.

Internal Representation and Noise Awareness

A key insight of the study is that internal recognition is fundamental to robustness. The t-SNE visualizations provide a clear demonstration of this effect.

t-SNE visualizations show that RAAT-trained models develop clearer internal clustering of different context types compared to standard models.

Standard models have "messy" internal representations where noise and golden contexts overlap. RAAT-trained models show much clearer clustering, especially for counterfactual noise, proving the model has "learned" to see the difference between factual and misleading context.

Conclusion

RAAT provides a robust framework for hardening LLMs against the inevitable noise of real-world retrieval systems. By focusing on the "maximum loss" during training and incorporating noise-awareness tasks, the model becomes significantly more resilient to irrelevant and counterfactual distractions. Future work will explore scaling these techniques to larger models and more diverse NLP tasks beyond open-domain question answering.

Summary updated

Abstract

Large Language Models (LLMs) exhibit substantial capabilities yet encounter challenges, including hallucination, outdated knowledge, and untraceable reasoning processes. Retrieval-augmented generation (RAG) has emerged as a promising solution, integrating knowledge from external databases to mitigate these challenges. However, inappropriate retrieved passages can potentially hinder the LLMs' capacity to generate comprehensive and high-quality responses. Prior RAG studies on the robustness of retrieval noises often confine themselves to a limited set of noise types, deviating from real-world retrieval environments and limiting practical applicability. In this study, we initially investigate retrieval noises and categorize them into three distinct types, reflecting real-world environments. We analyze the impact of these various retrieval noises on the robustness of LLMs. Subsequently, we propose a novel RAG approach known as Retrieval-augmented Adaptive Adversarial Training (RAAT). RAAT leverages adaptive adversarial training to dynamically adjust the model's training process in response to retrieval noises. Concurrently, it employs multi-task learning to ensure the model's capacity to internally recognize noisy contexts. Extensive experiments demonstrate that the LLaMA-2 7B model trained using RAAT exhibits significant improvements in F1 and EM scores under diverse noise conditions. For reproducibility, we release our code and data at: https://github.com/calubkk/RAAT.
BibTeX
@inproceedings{fang2024enhancing,
  author = {Feiteng Fang and Yuelin Bai and Shiwen Ni and Min Yang and Xiaojun Chen and Ruifeng Xu},
  title = {{Enhancing Noise Robustness of Retrieval-Augmented Language Models with Adaptive Adversarial Training}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2024},
  doi = {10.18653/v1/2024.acl-long.540},
}
Figure