For decades, Information Retrieval (IR) has followed a "retrieve-then-rank" paradigm. To find a document, a system first searches an external index—like an inverted index for keywords or a vector database for embeddings—and then re-ranks the results. However, a new direction led by researchers like Dara Bahri, Yi Tay, and Donald Metzler proposes collapsing this entire pipeline into a single neural network.
This approach, known as Generative Retrieval (GR) or the Differentiable Search Index (DSI), treats the search process as a sequence-to-sequence task. Instead of calculating the distance between vectors in a high-dimensional space, the model learns to "generate" the unique identifier of a relevant document directly from its internal parameters.
The Model is the Index
In traditional neural retrieval, models like Dual Encoders map queries and documents into a shared embedding space. Retrieval is then a matter of finding the "nearest neighbors" in an external database. As Jing Yao and Ji-Rong Wen argue in their work on DynamicRetriever, this creates a disconnect between the model’s internal representations and the static external index.
Generative Retrieval eliminates this disconnect by "internalizing" the corpus. During a "memorization" or indexing phase, the model (typically a Transformer) is trained to map document content to a unique Document Identifier (DocID). At inference time, when a user provides a query, the model autoregressively decodes the DocID of the most relevant item. As Qi Chen demonstrates with the Neural Corpus Indexer (NCI), this end-to-end differentiability allows the model to capture deep query-document interactions that traditional disjointed pipelines miss.
The Power of Semantic Identifiers
The success of a Differentiable Search Index depends heavily on how documents are named. If DocIDs are random integers, the model must memorize an arbitrary mapping. However, if DocIDs carry semantic meaning, the model can generalize better.
Researchers have explored several strategies for constructing these identifiers:
- Hierarchical Clustering: CodeDSI and TIGER use semantic clustering to create "Semantic IDs"—hierarchical tuples of codewords that represent an item's position in a conceptual tree.
- Natural Language Titles: James Thorne shows that using Wikipedia titles as DocIDs allows the model to treat retrieval as a translation task, leveraging the pre-existing linguistic knowledge of the Transformer.
- Relevance-based Indexing: Hansi Zeng and Chen Luo suggest that IDs should be optimized for the retrieval objective itself, leading to significant performance gains on large-scale benchmarks.
Scaling and Dynamic Updates
A common critique of Generative Retrieval is its perceived inability to handle dynamic corpora. If the "index" is stored in the model's weights, does the model need to be retrained every time a new document is added?
Recent work has addressed this "staleness" problem. Sanket Vaibhav Mehta introduced DSI++, which uses continual learning techniques like Generative Memory to update the model without forgetting old documents. Similarly, Varsha Kishore and Kilian Q Weinberger proposed IncDSI, which allows for document additions in milliseconds by formulating the update as a constrained optimization problem.
While scaling to millions of passages remains a challenge, Kai Hui and Ronak Pradeep have shown that synthetic query generation—training the model on queries it might see—is a critical component for making GR competitive at scale.
Beyond Text: Universal Multimodal Search
The generative paradigm is rapidly expanding beyond text retrieval into images, audio, and even tool-calling for AI agents.
- Visual Retrieval: Hao Sun (IRGen) and Yongqi Li (GRACE) have adapted GR for image retrieval, training models to "memorize" images and recall them via text identifiers.
- Audio and 3D: Frameworks like GTA Minghui Fang for audio and work by Chahine-Nicolas Zede for 3D point clouds demonstrate that generative retrieval offers constant-time complexity ($O(1)$) relative to the database size, as the cost is determined by the length of the ID rather than the number of candidates.
- Tool Use: Renxi Wang introduced ToolGen, which treats thousands of external APIs as unique tokens in a model's vocabulary, allowing an LLM to "retrieve" and "call" a tool in a single generative step.
By moving the index from an external database into the model's own memory, Generative Retrieval offers a path toward more unified, efficient, and semantically aware information systems.