TL;DR
HVMap is a lightweight compatibility layer that enables federated retrieval across multiple vector databases (vectorDBs) built with different, incompatible embedding models. By learning bidirectional mappings using a novel "fuzzy contrastive loss," it allows for accurate cross-database ranking without the prohibitive cost of re-embedding or re-indexing entire corpora, achieving up to 3.59× higher accuracy and 2.01× higher throughput than existing baselines.
What Problem Does This Paper Address?
In modern AI infrastructure, vector databases are the backbone of semantic search and Retrieval-Augmented Generation (RAG). However, real-world deployments often end up with a "fragmented" landscape:
- Model Evolution: Upgrading from an older model (like BERT) to a newer one (like GPT-based embeddings) often renders legacy indexes incompatible with new ones.
- Domain Specialization: Different departments might use specialized encoders for financial, biomedical, or scientific data.
When a single query needs to search across these heterogeneous databases, we hit a wall: vectors and similarity scores from different models are not directly comparable. The "brute force" solution—re-embedding and re-indexing everything into a unified space—is often too expensive or operationally disruptive for billion-scale datasets. Existing "fusion" methods (like simply merging top-K results) often fail because they don't account for the different score distributions of each model.
Methods
The authors propose HVMap (Heterogeneous Vector Mapping), which treats federated retrieval as a systems problem rather than just a representation learning one.
1. Bidirectional Mapping with Fuzzy Contrastive Loss
Instead of moving data, HVMap learns to "translate" queries and retrieved vectors between spaces. The core innovation is the Fuzzy Contrastive Loss. Standard contrastive learning treats all "negative" pairs the same. HVMap’s "fuzzy" approach uses an adaptive margin based on the actual semantic distance in the target embedding space. This ensures that the mapping preserves the specific "score profile" that the target vectorDB expects, leading to much more reliable global ranking.
2. Hub-and-Spoke Architecture
To avoid the $O(N^2)$ complexity of training mappings between every possible pair of $N$ databases, HVMap uses a hub-and-spoke design. One embedding space acts as the "hub." Queries are encoded once in the hub space and then propagated to "spoke" spaces. This reduces the number of required mapping functions to $O(N)$ and allows for efficient batch processing.
3. Dual-Space Ranking
When a candidate is retrieved from a database, HVMap calculates a final score using two signals: its similarity in its native space (error-free) and its similarity in the mapped space (providing cross-space evidence). The authors found that a 70/30 weight split ($w_1=0.7$ for native) consistently provided the best results.
Datasets
The researchers evaluated HVMap across several settings:
- Open-Domain: Six datasets from the BEIR benchmark (including Msmarco, Nq, and Hotpotqa).
- Cross-Domain: They constructed a new benchmark for heterogeneous cross-domain federation using an LLM-based pipeline to generate queries that require information from Financial (FIQA), Biomedical (BioASQ), and Scientific (SciDocs/SciFact) corpora simultaneously.
- Scale-Out: Experiments scaling up to six different vectorDBs at the 1-billion vector scale.
Key Insights and Ideas
- The "Score Compatibility" Challenge: The paper identifies that the primary hurdle isn't just finding the right vectors, but making their scores comparable so a global ranker can pick the true top-K.
- Preserving Autonomy: HVMap allows databases to remain physically separate and managed independently. This is crucial for organizations where different teams "own" different data silos.
- Efficiency of Lightweight Hubs: Using a lightweight model (like MiniLM) as the hub significantly boosts query throughput without a major sacrifice in accuracy, as the mapping functions can compensate for the hub's lower capacity.
- Fuzzy vs. Rigid Margins: By using the target space's own geometry to define "closeness" during training (the "fuzzy" part), the mapping function learns the nuances of the target model's behavior rather than just trying to mimic it perfectly.
Key Results
HVMap occupies a "sweet spot" in the quality-cost-throughput trade-off space, especially at scale.
- Accuracy: Achieved up to 3.59× higher nDCG@10 compared to non-migration baselines.
- Construction Cost: At the 1-billion vector scale, HVMap's setup time was as low as 0.3% of the time required for full migration (MIG). While MIG costs grow linearly with database size, HVMap’s cost is effectively constant once the mapping is trained.
- Throughput: Improved query throughput by up to 2.01× over methods that require running multiple full embedding models per query.
Limitations and Future Work
- Paired Data Requirement: Training the mapping functions requires "paired" embeddings (the same text passed through both models). This assumes the system designer has access to both embedding models or their APIs.
- Fixed Weighting: While $w_1=0.7$ worked well across their tests, the authors acknowledge that optimal weights might vary in massive federations with very different model qualities.
- Modality: The current work focuses on text. Extending this to cross-modal retrieval (e.g., searching image databases with text queries across different models) is a promising but unexplored direction.
Summary updated