In traditional neural retrieval, we treat a document like a single point in a high-dimensional semantic space. We take a complex text—perhaps a Wikipedia article covering history, geography, and politics—and compress it into a single fixed-length vector. This "Bi-encoder" approach is incredibly fast, but it suffers from what Shunyu Zhang and colleagues call a semantic bottleneck: the mathematical "over-squashing" of diverse topics into a single average representation.
When you average the vector for "quantum physics" and "baking," you don't get a representation of both; you often get a point in the middle of the vector space that represents neither. To solve this, a growing body of research suggests we should stop treating documents as points and start treating them as sets of vectors.
The Semantic Bottleneck and the Multi-Vector Shift
The core problem with single-vector embeddings is injective limitation. As Xuri Ge and Joemon M. Jose argue in their work on long-text retrieval, a single mathematical point cannot capture the multi-faceted nature of a long-form narrative.
Early intuition for this shift came from Hayato Kobayashi, who proposed treating documents as distributions of word embeddings rather than a single aggregate. By preserving the internal "cloud" of meanings, models can perform more granular matching. This idea has evolved into "Multi-View" or "Multi-Vector" representations, where a single document is indexed using multiple entry points, each capturing a different "facet" or "aspect" of the content.
How to Build a Multi-View Representation
Researchers have identified several distinct ways to "unfold" a document into multiple vectors:
1. Leveraging Internal Model Structure
Modern Transformers already process information in parallel through multi-head attention. Ales Kubicek and the team behind Multi-Head RAG (MRAG) demonstrated that different attention heads naturally specialize in different aspects of a query. By using the activations from multiple heads as distinct embeddings, they can solve "multi-aspect" queries that require information from semantically diverse sources. This is supported by Marym Teimouri, whose analysis confirmed that attention heads specialize in distinct semantic topics at later layers.
2. Synthetic Queries and Hypothetical Aspects
Another approach is to ask: "What questions could this document answer?" Zehan Li and Nan Yang developed a Dual Cross Encoder that generates multiple document representations based on synthetic "pseudo-queries." Similarly, the QuOTE framework by Andrew Neeser augments chunks with embeddings of hypothetical questions. This creates a "one-to-many" mapping in the vector database, providing multiple semantic pathways to reach the same source material.
3. Structural and Topical Decomposition
For long documents, the "dilution" of signals is a major hurdle. Minghan Li introduced BReps, which segments documents into blocks and encodes them independently to preserve local relevance. Alternatively, Hongyu Gong used Singular Value Decomposition (SVD) to extract hidden topic vectors, allowing a long document to be represented as a set of basis vectors that define its thematic range.
Bridging the Gap Between Speed and Accuracy
The multi-vector approach serves as a middle ground between two extremes:
- Bi-encoders: Fast (dot product) but lose detail.
- Cross-encoders: Accurate (token-level interaction) but too slow for large-scale search.
Models like the Poly-encoder, introduced by Jason Weston and Kurt Shuster, bridge this gap by representing the context as a small set of vectors that the candidate can attend to. Similarly, Giulio Zhou and Jacob Devlin proposed Multi-Vector Attention (MVA), which uses a small matrix of vectors to enable fine-grained interaction during re-ranking without the full cost of a Cross-encoder.
Why Multi-Vector Retrieval Works
The intuition behind the success of these models—seen in tasks ranging from scientific paper similarity to entity linking—is that they allow for asymmetric matching.
A user's query is usually specific, while a document is broad. By representing the document as a collection of views (e.g., its summary, its entities, its methodology, and its potential questions), the retriever can find the specific "view" that matches the query's intent. As Kuicai Dong showed with MC-indexing, matching against different levels of abstraction (raw text vs. keywords vs. summaries) can boost retrieval recall by over 40%.
By moving from a "point-to-point" matching paradigm to a "point-to-set" paradigm, multi-vector systems effectively dissolve the semantic bottleneck, allowing models to navigate the complex, multi-faceted landscape of human knowledge.