Abstract
NeuroGenPoisoning is a sophisticated attack framework designed to compromise Retrieval-Augmented Generation (RAG) systems by overriding a Large Language Model's (LLM) internal parametric memory. By identifying "Poison-Responsive Neurons" (PRNs) sensitive to external context, the framework employs genetic algorithms to evolve adversarial passages. Evaluation across SQuAD 2.0, TriviaQA, and WikiQA benchmarks demonstrates a Population Overwrite Success Rate (POSR) exceeding 90% against models like LLaMA-2 and Vicuna, while preserving text fluency and stealth.
Introduction
Retrieval-Augmented Generation (RAG) has become the industry standard for grounding Large Language Models (LLMs) in up-to-date, external information. However, this reliance introduces a critical vulnerability: knowledge poisoning. When external data contradicts a model's internal training data, a "knowledge conflict" occurs. Traditional attacks often fail because they rely on simple templates and do not account for the internal mechanisms LLMs use to resolve these conflicts. NeuroGenPoisoning addresses this by targeting the model's internal representation dynamics, ensuring that poisoned external knowledge consistently overrides internal "beliefs."
Methodology: Exploiting Internal Gatekeepers
The NeuroGenPoisoning framework operates through a multi-stage process that combines interpretability techniques with evolutionary computation.
- Identifying Poison-Responsive Neurons (PRNs): Using a technique called Integrated Gradients (IG), the authors identify a small subset of neurons (often just the top 10) that act as "gatekeepers," determining when the model should trust external context over its own memory.
- Adversarial Seed Generation: Initial "seed" passages containing target misinformation are generated using high-capability models like GPT-4 to ensure they appear as legitimate Wikipedia articles or news reports.
- Genetic Optimization: These seeds are evolved using a genetic algorithm where "fitness" is defined by the activation strength of the identified PRNs rather than surface-level text features.
- Neuron-Guided Evolution: The process uses internal neuron attribution signals to refine the population, ensuring that even unsuccessful passages contributing to the right internal circuits are preserved for future generations.
Experimental Evaluation
The framework was evaluated across three major open-domain Question Answering (QA) benchmarks: SQuAD 2.0, TriviaQA, and WikiQA. The target models included popular open-source architectures such as LLaMA-2-7b, Vicuna-7b/13b, and Gemma-7b.
Key Results
The optimization process proved highly effective, nearly doubling the success rate of initial seed passages within just 10 generations.
- High Success Rate: The method achieved a POSR of over 90% across all tested models and datasets.
- Stealth and Fluency: Evolved passages maintained high fluency, as measured by perplexity metrics, making them difficult to detect via standard automated filters or human review.
- Overcoming Strong Internal Memory: The attack successfully forced models to output incorrect answers even when the model had a strong internal preference for the correct fact (e.g., changing the model's "belief" about historical figures).
Conclusion and Future Work
NeuroGenPoisoning exposes a significant vulnerability in RAG systems by demonstrating how internal model interpretability can be weaponized to bypass safety and accuracy guardrails. While currently a white-box attack requiring access to model gradients, it highlights the urgent need for "neuron-aware" defensive strategies. Future research will focus on black-box transferability—using open models to attack proprietary ones—and the development of robust filters capable of detecting these optimized adversarial passages.
Summary updated