In the current era of Information Retrieval (IR), "dense retrieval" is the dominant paradigm. By encoding documents and queries into a shared vector space, models can capture semantic nuances that traditional keyword-based methods like BM25 miss. However, as we scale these systems from small research benchmarks to production-grade indices containing millions or billions of documents, a fundamental problem emerges: the performance of single-vector embeddings begins to collapse.
This phenomenon isn't just a matter of needing more training data or larger models. It represents a fundamental geometric bottleneck in how we represent information.
The Scaling Paradox
On small datasets, dense retrievers often outperform lexical search. But as Nils Reimers and Iryna Gurevych demonstrate in The Curse of Dense Low-Dimensional Information Retrieval for Large Index Sizes, this advantage evaporates as the index grows. They found that dense models experience a "tipping point" where the probability of irrelevant documents falling into the retrieval radius of a query increases exponentially.
In a massive search space, the vector environment becomes overcrowded. When you have millions of documents, the "semantic neighborhood" of a query becomes so densely packed that the model can no longer distinguish between a true semantic match and a "false positive" that happens to be nearby in the vector space. This is why, as Zoran Medic notes in Large-scale Evaluation of Transformer-based Article Encoders on the Task of Citation Recommendation, traditional lexical search (BM25) often remains superior at scale; keywords provide a sparse, high-precision signal that doesn't suffer from the "crowding" of dense latent spaces.
The Combinatorial Explosion
Why does this crowding happen? The answer lies in the relationship between the dimensionality of the vector space and the complexity of the queries we ask.
In On the Theoretical Limitations of Embedding-Based Retrieval, Orion Weller, Michael Boratko, and their colleagues prove that single-vector embeddings face a hard mathematical ceiling. As queries become more complex—for example, requiring a document to satisfy multiple logical conditions—the number of possible "relevant subsets" of documents grows combinatorially.
Imagine a corpus of documents. If a query asks for documents that are about "Biology" AND "Published in 2022" AND "Peer-reviewed," the retriever must be able to isolate that specific intersection. As you add more attributes or logical constraints, the number of potential document combinations you might need to retrieve grows much faster than the geometric capacity of a fixed-dimensional vector space. Eventually, the space simply runs out of "room" to keep these different subsets distinct. Distinct sets of documents become indistinguishable in the vector space, leading to unavoidable retrieval failures regardless of how much the model is trained.
Capacity versus Learnability
There is a nuanced debate about whether this bottleneck is a matter of space (approximability) or training (learnability).
In $\mathbb{R}^{2k}$ is Theoretically Large Enough for Embedding-based Top-$k$ Retrieval, Zihao Wang and colleagues argue that the geometric bottleneck might be smaller than we think. They prove that to perfectly retrieve any subset of $k$ items, you theoretically only need a dimension of $2k$, independent of the total corpus size.
This suggests that the "ceiling" identified by Weller might not be an absolute lack of geometric volume, but rather a failure of our current architectures to learn how to pack information that efficiently. However, even if the space is theoretically large enough, the practical reality remains: as the index grows, the "margin" between documents shrinks.
The Impact of Noise and Precision
This crowding makes the system incredibly fragile. In Knowledge Base Index Compression via Dimensionality and Precision Reduction, Vilém Zouhar and co-authors show that as indices reach the scale of billions, they become hypersensitive to noise.
When a vector space is sparsely populated, you can compress vectors (using PCA or quantization) without much loss in performance because there is plenty of "empty space" between document points. But in a crowded, large-scale index, even a tiny bit of noise from quantization can push a document vector across the boundary of a query's retrieval radius, causing a ranking error. The "geometric ceiling" thus creates a double bind: we need high dimensions to separate documents, but high dimensions at scale make the index too large to store without compression, and compression destroys the precision needed to navigate the crowded space.
Summary of the Challenge
The research across these papers points to a unified conclusion: the "single vector per document" approach has a finite lifespan as data scales.
- Crowding: Large indices lead to an exponential increase in false positives as the vector space fills up.
- Geometry: Fixed dimensions cannot represent the combinatorial explosion of document subsets required by complex queries.
- Fragility: The lack of "spatial margin" in large indices makes them highly sensitive to the noise introduced by necessary compression techniques.
Moving forward, the field is forced to choose between increasing dimensionality (which is computationally expensive), moving toward multi-vector representations, or finding ways to hybridize dense retrieval with the robust, sparse signals of traditional lexical search.