Abstract
This study investigates the internal mechanisms of Dense Passage Retrieval (DPR) by comparing a fine-tuned BERT model with its pre-trained counterpart. Using mechanistic interpretability techniques—linear probing, knowledge attribution, and model editing—we demonstrate that DPR fine-tuning does not enhance the model's inherent ability to discriminate between passages. Instead, it decentralizes internal knowledge representations, creating multiple access pathways to existing information. Crucially, retrieval performance remains strictly bounded by the model's pre-existing parametric knowledge, as the removal of specific facts from the base model leads to a corresponding failure in retrieval.
Introduction
Retrieval-Augmented Generation (RAG) has emerged as a primary solution for mitigating hallucinations in Large Language Models (LLMs). By fetching relevant documents from an external corpus before generation, RAG provides a grounded context for the model. Dense Passage Retrieval (DPR) serves as the backbone for many of these systems, utilizing models like BERT to map queries and documents into a shared vector space.
Despite its empirical success, the internal dynamics of DPR fine-tuning remain poorly understood. This research seeks to answer a fundamental question: Does fine-tuning for retrieval teach the model new ways to identify relevance, or does it simply reorganize the model's existing internal knowledge? By looking "under the hood," we can determine if the retrieval process is truly discovering new relationships or merely reflecting the model's internal memory.
Methods
The researchers employed three primary mechanistic interpretability techniques to analyze a BERT-base model before and after DPR fine-tuning:
- Linear Probing: Simple linear classifiers were trained on the internal activations of frozen layers to detect if the signal required to distinguish correct passages from "hard negatives" (misleadingly similar but incorrect passages) already exists within the pre-trained features.
- Knowledge Attribution (Neuron Analysis): Using integrated gradients, the authors identified specific neurons contributing most to the model's output for a given query. This allows for a mapping of how "concentrated" or "distributed" a specific fact is within the network.
- Model Editing: To test the dependency of retrieval on internal knowledge, the authors used surgical editing tools (TransformerPatch, MalMen, and Mend) to delete specific facts from the pre-trained weights. They then observed whether the DPR-trained version could still retrieve documents associated with those deleted facts.
Key Insights: The Discrimination Signal and Decentralization
The Discrimination Signal is Pre-existing
A surprising finding of this study is that pre-trained BERT is already nearly as capable as DPR-BERT at distinguishing correct passages from incorrect ones when presented with them directly. Fine-tuning for retrieval does not appear to "sharpen" the model's discriminatory ability. Instead, it teaches the model how to project this pre-existing knowledge into a searchable vector space.
As shown in the probing results, in a 2-passage discrimination task, pre-trained BERT achieved an accuracy of 84% at the final layer, while the DPR-trained version reached 82%. This parity suggests that the ability to recognize relevance is an inherent property of the pre-training phase rather than a result of specialized retrieval tuning.
Knowledge Decentralization
The most significant change observed during fine-tuning is "knowledge decentralization." In a standard pre-trained model, a specific fact is often stored in a small cluster of highly precise neurons. DPR training spreads this information out, engaging a much larger number of neurons for the same query. This creates multiple "access pathways," making the model more robust to varying query formulations.
For example, as seen in the table above, a query regarding "Robinson projections" activated only 220 neurons in the pre-trained model but triggered 1,323 neurons in the DPR model. This quantitative "neuron explosion" illustrates the model's strategy of decentralizing knowledge to improve retrieval robustness.
The Internal Knowledge Bound
Retrieval is not a panacea for a model's lack of information. The study found that retrieval performance is strictly tethered to the model's internal weights. When a fact was successfully edited out of the pre-trained BERT model, the DPR version's ability to retrieve the relevant document collapsed 81% to 100% of the time, depending on the editing method used. This indicates that DPR can only effectively retrieve what the model already "knows" or has a representation for internally.
Conclusion and Future Directions
The findings suggest that DPR acts more as a knowledge reorganizer than a new knowledge acquirer. While it improves the accessibility of information through decentralization, it remains limited by the scope of the initial pre-training.
Future research should focus on exposing retrieval models to more diverse data during training to expand the range of information that can be decentralized. Additionally, there is a need for models that can signal uncertainty when a query falls outside their internal knowledge base, rather than retrieving the "least-wrong" document. Finally, bridging the gap between parametric (internal) and non-parametric (external) memory through direct neuron-to-knowledge mapping remains a promising frontier.
Summary updated