Dense retrieval models have transformed information retrieval by mapping text into continuous vector spaces, effectively bridging the "lexical gap" where keyword searches fail. However, as these systems scale from laboratory benchmarks to production-grade indexes containing millions or billions of documents, they encounter a fundamental bottleneck: the geometric capacity of the vector space itself.
The Scaling Ceiling
In a dense retrieval system, every document is compressed into a fixed-length vector (e.g., 768 dimensions). Retrieval is then a matter of finding the documents closest to a query vector. While this works exceptionally well for small datasets, Reimers et al. demonstrate that performance often degrades significantly as the index grows.
The intuition is one of "crowding." As more documents are added to a fixed-dimensional space, the probability that an irrelevant document is mapped closer to the query than the ground-truth document increases mathematically. Reimers et al. show that at production scales, the high density of false positives can cause sophisticated neural models to be outperformed by traditional, high-dimensional keyword methods like BM25.
Combinatorial Complexity and Sphere Packing
Why does this degradation happen? The problem is not just about the number of documents, but the number of potential query results. Weller et al. argue that single-vector embeddings face a geometric "ceiling." As a corpus grows, the number of possible subsets of relevant documents increases combinatorially. A fixed-dimensional space eventually lacks the "room" to maintain a distinct separation between all these possible result sets.
This can be understood through the lens of high-dimensional geometry. Pagh et al. analyze this using inner product filters and the overlap of "spherical caps." To distinguish between different vectors at a specific similarity threshold, a minimum number of bits (and thus dimensions) is required. If the dimensionality is too low, the "caps" representing different documents overlap so much that they become indistinguishable, leading to unavoidable retrieval failures. Weller et al. formalize this by showing that for any fixed dimension, there exists a critical corpus size beyond which the model mathematically cannot separate the correct documents from the noise.
The $2k$ Paradox: Capacity vs. Learnability
Interestingly, the relationship between dimensionality and corpus size is not a simple linear one. Wang et al. provide a surprising theoretical counterpoint: to perfectly retrieve any subset of $k$ items from a universe of $m$ elements, you theoretically only need a vector space of dimension $2k$, regardless of how large $m$ is.
This concept, termed the Minimal Embeddable Dimension (MED), suggests that the "geometric bottleneck" might not be a lack of physical space in the vectors. Instead, the authors argue that the failure of current models is a problem of learnability rather than approximability. While a $2k$-dimensional space could theoretically house the representations, our current training methods and architectures struggle to find the precise, highly-optimized mapping required to utilize that space effectively as the corpus scales.
Real-World Implications
This tension between theoretical capacity and practical performance is most visible in specialized domains. Ajith et al. introduced the LitSearch benchmark to test retrieval in scientific literature. They found that as the number of indexed papers grows to reflect the global scientific record, dense models struggle with an influx of "distractor" documents that look similar in vector space but are factually irrelevant.
The research suggests that solving the "curse of dimensionality" in retrieval will require more than just larger models; it requires addressing how we map complex information into finite spaces to avoid the inevitable collisions that occur when the world we are indexing grows faster than the dimensions we use to represent it.
Key Foundational Work
The Curse of Dense Low-Dimensional Information Retrieval for Large Index Sizes (2021) — contributed 6 concept elements to this direction.
The space complexity of inner product filters (2019) — contributed 1 concept elements to this direction.