Abstract
Dense Passage Retrieval (DPR) typically relies on Euclidean or cosine distance to measure query-passage relevance in embedding space, which is effective when embeddings lie on a linear manifold. However, our experiments across DPR benchmarks suggest that embeddings often lie on lower-dimensional, non-linear manifolds, especially in out-of-distribution (OOD) settings, where cosine and Euclidean distance fail to capture semantic similarity. To address this limitation, we propose a manifold-aware distance metric for DPR (MA-DPR) that models the intrinsic manifold structure of passages using a nearest neighbor graph and measures query-passage distance based on their shortest path in this graph. We show that MA-DPR outperforms Euclidean and cosine distances by up to 26% on OOD passage retrieval with comparable in-distribution performance across various embedding models while incurring a minimal increase in query inference time. Empirical evidence suggests that manifold-aware distance allows DPR to leverage context from related neighboring passages, making it effective even in the absence of direct semantic overlap. MADPR can be applied to a wide range of dense embedding and retrieval tasks, offering potential benefits across a wide spectrum of domains.
Introduction
Dense Passage Retrieval (DPR) is a cornerstone of modern information retrieval and Retrieval-Augmented Generation (RAG). It maps queries and documents into a high-dimensional vector space where semantic relevance is typically measured by Euclidean or Cosine distance. However, these metrics assume a flat, linear embedding space. This paper argues that embeddings often reside on lower-dimensional, non-linear manifolds—akin to a crumpled sheet of paper. In such "curved" spaces, standard "straight-line" metrics fail to capture true semantic similarity, particularly in out-of-distribution (OOD) scenarios.
Methodology: Manifold-Aware DPR
To address this, we propose MA-DPR (Manifold-Aware DPR). Instead of global Euclidean coordinates, MA-DPR models the intrinsic structure of the data using a nearest-neighbor graph. By shifting the focus from global coordinates to local connectivity, the algorithm can navigate the "curved" geometry of the embedding space.
The method consists of two phases:
- Offline Graph Construction: A $K$-Nearest Neighbor (KNN) graph is built where nodes represent passages. Edges are defined by proximity in the embedding space. We explored both standard Euclidean distance and Spectral Distance (using graph Laplacian eigenvectors) to define these edges.
- Online Ranking: Queries are dynamically connected to their $K$ nearest neighbors in the graph. Relevance is then determined by the shortest path from the query to any given passage node using Dijkstra’s algorithm.
We explored two edge weighting strategies: Distance Cost (DC), using actual embedding distances, and Uniform Cost (UC), which simply counts the number of "hops" between nodes.
Empirical Evidence of the Manifold Hypothesis
We empirically verified that in-distribution, Euclidean and manifold distances correlate strongly. However, in OOD settings, they diverge significantly. This divergence proves that standard DPR models struggle to maintain a linear representation of relevance when stepping outside their training data.
Key Results
We evaluated MA-DPR using two embedding models—tas-b (trained on MS MARCO) and SciNCL—across four benchmarks: MS MARCO (web search), ANTIQUE (community QA), NFCorpus (medical), and SciDocs (scientific).
- OOD Superiority: MA-DPR significantly outperformed standard DPR on OOD tasks, showing improvements of up to 26% in retrieval metrics like nDCG@20.
- No In-Distribution Penalty: On datasets the models were trained on (e.g., MS MARCO for tas-b), MA-DPR matched the performance of standard Euclidean distance, meaning there is no "tax" for using the more complex metric.
- The Bridge Effect: MA-DPR can retrieve relevant passages even without direct semantic overlap by traversing "intermediate" nodes that bridge conceptual gaps, effectively leveraging the context of the entire neighborhood.
- Efficiency: Despite the added complexity of graph traversal, the per-query latency overhead is minimal, adding only 2–4ms. This makes the approach highly practical for real-world production systems.
Conclusion
MA-DPR offers a robust alternative to standard distance metrics by accounting for the non-linear geometry of embedding spaces. By leveraging the local connectivity of the data manifold, it significantly enhances retrieval accuracy in diverse, out-of-distribution domains without sacrificing speed or in-distribution performance. Future work will explore scaling graph construction to billions of documents using approximate nearest neighbor methods.
Summary updated