Standard dense retrieval models operate on a simple paradigm: map both queries and documents into a shared vector space, and evaluate their relevance using a single geometric operation like a dot product. In this setup, a single vector—such as a BERT [CLS] token—must compress all the semantic nuances, topics, and facts of an entire document into a single fixed-length point.
For long, multi-topic, or highly technical documents, this compression creates a severe semantic bottleneck. As demonstrated by Weller et al. in On the Theoretical Limitations of Embedding-Based Retrieval, single-vector models face a fundamental geometric ceiling: they mathematically cannot represent every possible combination of relevant documents as a corpus grows. When forced to compress diverse facts into a single fixed-length embedding, models suffer from "over-squashing," causing retrieval systems to miss fine-grained details or sub-sections of a long text (Liu et al. in Towards Better Entity Linking with Multi-View Enhanced Distillation). This limitation explains why standard dense retrievers often underperform classical keyword-matching algorithms like BM25 on long-form or specialized domain content (Luan et al. in Sparse, Dense, and Attentional Representations for Text Retrieval; Luo et al. in Improving Biomedical Information Retrieval with Neural Retrievers).
To bypass this geometric bottleneck, a prominent direction in information retrieval replaces single-vector embeddings with multi-vector document representations. Instead of compressing a document into a single point, these architectures represent a document as a set, matrix, or distribution of multiple distinct vectors. By scaling the number of vectors rather than the dimensionality of a single vector, the model expands its representational capacity, preserving granular data and capturing diverse semantic facets simultaneously.
Strategies for Multi-Vector Decomposition
How do models break a document down into multiple vectors? The literature highlights three primary strategies:
- Token and View-Based Decomposition: Models can generate multiple vectors using distinct "viewer" tokens or attention heads. For instance, Zhang et al. in Multi-View Document Representation Learning for Open-Domain Dense Retrieval introduced a Multi-View Representation (MVR) framework that uses multiple viewer tokens to capture different semantic views of a document. Similarly, Teimouri et al. in A Deep Dive into Multi-Head Attention and Multi-Aspect Embedding showed that individual attention heads naturally specialize in distinct semantic topics, meaning a collection of vectors mapped to specific attention heads can preserve diverse features that global pooling would destroy.
- Aspect and Facet-Based Decomposition: Documents can be explicitly decomposed into vectors representing distinct categories of knowledge or structural attributes. For scientific literature, Ostendorff et al. in Specialized Document Embeddings for Aspect-based Similarity of Research Papers constructed separate embedding spaces for specific aspects like Task, Method, and Dataset, allowing high-precision matching on specific technical components. Risch et al. in Multifaceted Domain-Specific Document Embeddings and Pappas et al. in Explicit Document Modeling through Weighted Multiple-Instance Learning similarly leveraged category-specific or sentence-level weighted averages to form decomposable document vectors. This can extend to relational factors like topics and citation history (Zhang et al. in Chain-of-Factors Paper-Reviewer Matching).
- Mathematical and Distributional Generalizations: Rather than discrete vector sets, documents can be modeled as higher-order mathematical structures. Mishra et al. in Generalised Spherical Text Embedding proposed representing documents as multi-column matrices on a spherical manifold to capture multiple topics. Alternatively, Zamani et al. in Multivariate Representation Learning for Information Retrieval modeled documents as multivariate distributions, using variance to represent the breadth of information and allow a single representation to overlap with diverse query distributions.
Why It Works: Localized Attention and Dynamic Routing
The power of multi-vector representation lies in how retrieval is executed. In a single-vector system, retrieval relies on a global dot product, which forces a query to match the average meaning of the entire document.
In contrast, multi-vector systems utilize a localized attention or dynamic selection mechanism during retrieval (Luan et al.; Luo et al. in Neural Retriever and Go Beyond: A Thesis Proposal). When a query is processed, the retrieval engine computes similarities across the entire set of document vectors and dynamically selects or weights the most relevant vector (often using a MaxSim operator or localized attention).
Single-Vector Paradigm (Information Loss):
[Long Document] ---> [Compression Bottleneck] ---> Single Vector (Averaged Meaning)
|
Query (Fails to match diluted details)
Multi-Vector Paradigm (Granular Routing):
/---> Vector 1: Topic A (e.g., Symptoms)
[Long Document] ---> |----> Vector 2: Topic B (e.g., Diagnosis)
\---> Vector 3: Topic C (e.g., Treatment) <--- Query routes directly here
Consider a comprehensive medical document covering the symptoms, diagnosis, and treatments for a rare disease. A single-vector model averages these topics, diluting the specific details. If a user queries specifically about the treatment, the single-vector embedding might fail to match because the overall document vector is pulled toward the symptoms and diagnosis.
A multi-vector model, however, maintains distinct vectors for different facets. The query routes directly to the specific vector representing the treatment facet, completely bypassing the irrelevant sections. By decoupling the document's sub-topics, multi-vector representations achieve the high precision of cross-attentional models while maintaining the sub-linear search speeds required for large-scale production use.
Key Foundational Work
Sparse, Dense, and Attentional Representations for Text Retrieval (2020) — contributed 3 concept elements to this direction.
Explicit Document Modeling through Weighted Multiple-Instance Learning (2017) — contributed 2 concept elements to this direction.
SCDV : Sparse Composite Document Vectors using soft clustering over distributional representations (2017) — contributed 1 concept elements to this direction.