Abstract
Dense Passage Retrieval (DPR) is a critical component in Retrieval-Augmented Generation (RAG) systems, yet the internal mechanisms by which it improves retrieval performance remain poorly understood. This paper investigates whether DPR training facilitates the acquisition of new factual knowledge or merely reorganizes existing internal representations. Using mechanistic interpretability techniques, including linear probing, knowledge attribution, and model editing, we demonstrate that DPR training "decentralizes" knowledge by recruiting a significantly larger number of neurons to represent specific facts. However, we find that DPR is strictly bounded by the model's pre-trained internal knowledge; surgically removing facts from the base model renders them un-retrievable by the DPR-trained version. These findings suggest that DPR functions primarily as a specialized alignment mechanism rather than a knowledge acquisition process.
Introduction
Retrieval-Augmented Generation (RAG) has emerged as a primary strategy for mitigating hallucinations in Large Language Models (LLMs) by grounding outputs in external evidence. The effectiveness of RAG relies heavily on Dense Passage Retrieval (DPR), where bi-encoders (typically BERT-based) are fine-tuned to map queries and relevant documents into a shared vector space. While the performance gains of DPR are well-documented, the nature of the internal changes during fine-tuning—specifically whether the model learns new facts or simply optimizes the accessibility of pre-existing ones—remains an open question.
Methodology
To analyze the internal dynamics of BERT-base during DPR training, we employed a three-pronged mechanistic approach:
- Linear Probing: We evaluated the model's inherent discriminative ability by training simple classifiers on internal activations to distinguish between relevant passages and "hard negatives."
- Knowledge Attribution: We identified "knowledge neurons" within the feedforward layers that contribute most to specific outputs, measuring the change in the number of strongly activated neurons before and after DPR training.
- Model Editing (Knowledge Removal): Using surgical editing tools—TransformerPatch, MalMen, and Mend—we deleted specific facts from the pre-trained BERT model to observe if the DPR-trained version could still retrieve the associated information.
Knowledge Decentralization and Neuron Activation
A key finding of our analysis is the phenomenon of "knowledge decentralization." In standard pre-trained models, factual information is often localized within a narrow set of neurons. DPR training significantly expands this representation, recruiting a much larger population of neurons to encode the same information. This process creates multiple "access pathways" to the same data, making the retrieval process more robust to variations in query phrasing.
As shown in Table 5, the number of strongly activated neurons increases dramatically following DPR training. For instance, a query regarding distortion in map projections saw an increase from 220 activated neurons in the pre-trained model to 1,323 neurons in the DPR-trained version.
The Internal Knowledge Bound
Perhaps the most significant insight is that DPR is fundamentally constrained by the model's initial pre-training. Our "deleted fact" experiments revealed that if a specific piece of information is removed from the pre-trained base model, the DPR-trained model loses the ability to retrieve it. This indicates that DPR does not learn to "find" information in an external corpus in the traditional sense; rather, it learns to align external documents with the internal knowledge base established during pre-training.
Quantitative results from the model editing experiments (Table 6) show that between 81% and 100% of facts removed from the pre-trained model were subsequently un-retrievable by the DPR model. This confirms that the "retrieval" process is essentially a reorganization of internal memory.
Conclusion
Our study demonstrates that Dense Passage Retrieval does not "teach" models new information. Instead, it decentralizes existing knowledge, making it more accessible through vector-based retrieval. Because the model's retrieval capabilities are strictly bounded by its pre-trained internal knowledge, future work should focus on methods for injecting new facts directly into decentralized representations or developing models that can explicitly recognize the limits of their internal knowledge bases.
Summary updated