Dense Passage Retrieval: Is it Retrieving?

Benjamin Z. Reichman, Larry Heck

Abstract

This study investigates the internal mechanisms of Dense Passage Retrieval (DPR) by comparing a fine-tuned BERT model with its pre-trained counterpart. Using mechanistic interpretability techniques—linear probing, knowledge attribution, and model editing—we demonstrate that DPR fine-tuning does not enhance the model's inherent ability to discriminate between passages. Instead, it decentralizes internal knowledge representations, creating multiple access pathways to existing information. Crucially, retrieval performance remains strictly bounded by the model's pre-existing parametric knowledge, as the removal of specific facts from the base model leads to a corresponding failure in retrieval.

Introduction

Retrieval-Augmented Generation (RAG) has emerged as a primary solution for mitigating hallucinations in Large Language Models (LLMs). By fetching relevant documents from an external corpus before generation, RAG provides a grounded context for the model. Dense Passage Retrieval (DPR) serves as the backbone for many of these systems, utilizing models like BERT to map queries and documents into a shared vector space.

Despite its empirical success, the internal dynamics of DPR fine-tuning remain poorly understood. This research seeks to answer a fundamental question: Does fine-tuning for retrieval teach the model new ways to identify relevance, or does it simply reorganize the model's existing internal knowledge? By looking "under the hood," we can determine if the retrieval process is truly discovering new relationships or merely reflecting the model's internal memory.

Methods

The researchers employed three primary mechanistic interpretability techniques to analyze a BERT-base model before and after DPR fine-tuning:

  1. Linear Probing: Simple linear classifiers were trained on the internal activations of frozen layers to detect if the signal required to distinguish correct passages from "hard negatives" (misleadingly similar but incorrect passages) already exists within the pre-trained features.
  2. Knowledge Attribution (Neuron Analysis): Using integrated gradients, the authors identified specific neurons contributing most to the model's output for a given query. This allows for a mapping of how "concentrated" or "distributed" a specific fact is within the network.
  3. Model Editing: To test the dependency of retrieval on internal knowledge, the authors used surgical editing tools (TransformerPatch, MalMen, and Mend) to delete specific facts from the pre-trained weights. They then observed whether the DPR-trained version could still retrieve documents associated with those deleted facts.

Key Insights: The Discrimination Signal and Decentralization

The Discrimination Signal is Pre-existing

A surprising finding of this study is that pre-trained BERT is already nearly as capable as DPR-BERT at distinguishing correct passages from incorrect ones when presented with them directly. Fine-tuning for retrieval does not appear to "sharpen" the model's discriminatory ability. Instead, it teaches the model how to project this pre-existing knowledge into a searchable vector space.

Table 1: Probing accuracy across layers for various passage discrimination tasks. Note the similarity between pre-trained and DPR-trained models in the final layers.

As shown in the probing results, in a 2-passage discrimination task, pre-trained BERT achieved an accuracy of 84% at the final layer, while the DPR-trained version reached 82%. This parity suggests that the ability to recognize relevance is an inherent property of the pre-training phase rather than a result of specialized retrieval tuning.

Knowledge Decentralization

The most significant change observed during fine-tuning is "knowledge decentralization." In a standard pre-trained model, a specific fact is often stored in a small cluster of highly precise neurons. DPR training spreads this information out, engaging a much larger number of neurons for the same query. This creates multiple "access pathways," making the model more robust to varying query formulations.

Table 2: Comparison of activated neurons and retrieval performance between Pre-trained and DPR-trained BERT.

For example, as seen in the table above, a query regarding "Robinson projections" activated only 220 neurons in the pre-trained model but triggered 1,323 neurons in the DPR model. This quantitative "neuron explosion" illustrates the model's strategy of decentralizing knowledge to improve retrieval robustness.

The Internal Knowledge Bound

Retrieval is not a panacea for a model's lack of information. The study found that retrieval performance is strictly tethered to the model's internal weights. When a fact was successfully edited out of the pre-trained BERT model, the DPR version's ability to retrieve the relevant document collapsed 81% to 100% of the time, depending on the editing method used. This indicates that DPR can only effectively retrieve what the model already "knows" or has a representation for internally.

Conclusion and Future Directions

The findings suggest that DPR acts more as a knowledge reorganizer than a new knowledge acquirer. While it improves the accessibility of information through decentralization, it remains limited by the scope of the initial pre-training.

Future research should focus on exposing retrieval models to more diverse data during training to expand the range of information that can be decentralized. Additionally, there is a need for models that can signal uncertainty when a query falls outside their internal knowledge base, rather than retrieving the "least-wrong" document. Finally, bridging the gap between parametric (internal) and non-parametric (external) memory through direct neuron-to-knowledge mapping remains a promising frontier.

Summary updated

Abstract

Dense passage retrieval (DPR) is the first step in the retrieval augmented generation (RAG) paradigm for improving the performance of large language models (LLM). DPR fine-tunes pre-trained networks to enhance the alignment of the embeddings between queries and relevant textual data. A deeper understanding of DPR fine-tuning will be required to fundamentally unlock the full potential of this approach. In this work, we explore DPR-trained models mechanistically by using a combination of probing, layer activation analysis, and model editing. Our experiments show that DPR training decentralizes how knowledge is stored in the network, creating multiple access pathways to the same information. We also uncover a limitation in this training style: the internal knowledge of the pre-trained model bounds what the retrieval model can retrieve. These findings suggest a few possible directions for dense retrieval: (1) expose the DPR training process to more knowledge so more can be decentralized, (2) inject facts as decentralized representations, (3) model and incorporate knowledge uncertainty in the retrieval process, and (4) directly map internal model knowledge to a knowledge base.
BibTeX
@inproceedings{reichman2024dense,
  author = {Benjamin Z. Reichman and Larry Heck},
  title = {{Dense Passage Retrieval: Is it Retrieving?}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2024},
  doi = {10.18653/v1/2024.findings-emnlp.791},
}
Figure