In traditional information retrieval, searching a corpus is a two-step process of representation and matching. Dual-encoder architectures map queries and documents into a shared continuous vector space, relying on vector similarity (like dot product) to find relevant texts.
Generative Retrieval (GR) collapses this pipeline into a single sequence-to-sequence task. Instead of calculating similarities across millions of vectors, a single autoregressive language model ingests a user query and directly generates the unique identifier (a docid) of the relevant document.
Early generative retrieval frameworks assigned arbitrary numeric strings or hierarchical cluster codes (e.g., [doc_48291]) as docids. However, forcing a language model to output abstract symbols creates a severe friction point: it demands pure rote memorization, treating the model like a digital cassette tape rather than a semantic engine. To unlock the true potential of generative retrieval, a rapidly growing body of research replaces these artificial symbols with natural language identifiers—such as URLs, semantic summaries, or document-constrained n-grams.
The Friction of Arbitrary Identifiers
Why do numeric or arbitrary identifiers fail? The answer lies in how large language models store and access information.
As Qian Liu et al. demonstrate in Beyond Memorization: The Challenge of Random Memory Access in Language Models, language models struggle with non-sequential, random memory access. They locate specific information within their parameter space using high-dimensional semantic associations rather than structured index tables. When a model is forced to map a natural language query to an arbitrary numeric token sequence, it cannot use any of its pre-trained linguistic knowledge. The model must learn an entirely new, out-of-distribution mapping from scratch.
This reliance on rote memorization causes two major issues:
- Poor Generalization: The model overfits to the exact document-ID mappings seen during training.
- The Dynamic Corpus Problem: As explored by Zhen Zhang et al. in Replication and Exploration of Generative Retrieval over Dynamic Corpora, when new documents are added to a corpus, numeric identifiers completely break down. Because a new numeric ID carries zero semantic context, the model cannot generalize to unseen documents without expensive, full-corpus retraining.
The Anatomy of a Semantic Identifier
By replacing arbitrary numbers with human-readable, semantically rich strings, researchers align the retrieval target with the model's pre-trained internal representations. The retrieval task shifts from blind memorization to a guided semantic mapping process.
Several distinct strategies have emerged for constructing these natural language identifiers:
1. Tokenized URLs
Web URLs are highly structured yet human-readable strings that frequently mirror the underlying topic of a page. Noah Ziems et al. discovered in Large Language Models are Built-in Autoregressive Search Engines that pre-trained LLMs can achieve remarkable retrieval accuracy out-of-the-box simply by prompting them to generate Wikipedia URLs. Because the linguistic structure of a URL correlates with the query intent, the model can navigate the document space using its built-in language understanding.
To formalize this within generative retrieval, Ruiyang Ren et al. introduced TOME: A Two-stage Approach for Model-based Retrieval, which tokenizes URLs as docids. This bridges the gap between the retrieval target and the natural language patterns the model encountered during large-scale pre-training.
2. Semantic Summaries and Elaborative Descriptions
Another approach is to condense the document into a short, highly descriptive natural language sentence or summary. In UniGen: A Unified Generative Framework for Retrieval and Question Answering with Large Language Models, Xiaoxi Li et al. utilize an LLM to generate semantic summaries that serve as docids.
Similarly, Yubao Tang et al. developed SE-DSI in Semantic-Enhanced Differentiable Search Index Inspired by Learning Strategies, replacing abstract IDs with "Elaborative Descriptions." When a user inputs a query, the model generates these descriptive targets, leveraging its pre-existing linguistic associations to match user intent with the document's core essence. Yujia Zhou et al. further optimized this paradigm in Enhancing Generative Retrieval with Reinforcement Learning from Relevance Feedback by using reinforcement learning to align the generation of these human-readable strings directly with search relevance signals.
3. Lexical Keywords and N-Grams
Instead of full summaries, documents can be identified by distinctive sequences of words extracted directly from the text. Sunkyung Lee 0001 et al. proposed GLEN in GLEN: Generative Retrieval via Lexical Index Learning, which assigns dynamic lexical keywords as identifiers that evolve during training.
Taking this a step further, Shanbao Qiao et al. introduced DiffusionRet in DiffusionRet: Diffusion-Enhanced Generative Retriever using Constrained Decoding, which forces the model to decode specific n-grams guaranteed to exist within the document collection. As also leveraged by Jiehan Cheng et al. in Descriptive and Discriminative Document Identifiers for Generative Retrieval, this strategy ensures that the model operates entirely within the natural language domain, minimizing the discrepancy between pre-training and fine-tuning.
Why It Works: Semantic Anchoring
The fundamental reason natural language identifiers outperform numeric ones is a mechanism known as semantic anchoring.
When an identifier consists of real words, it acts as a conceptual anchor within the model's high-dimensional embedding space. For instance, in biomedical text processing, Zhijian Chen et al. show in GeneSUM: Large Language Model-based Gene Summary Extraction that transforming structured metadata into natural language descriptions provides stable target representations that guide information extraction.
Even in context-window management, as Meiru Zhang demonstrates in Can We Instruct LLMs to Compensate for Position Bias?, explicit textual labels serve as semantic anchors that allow a model to navigate and prioritize specific information blocks within a long sequence.
[Arbitrary ID Paradigm]
Query: "What causes seasonal allergies?" ──(Rote Memorization)──> Target: "ID_84920" (No semantic link)
[Natural Language ID Paradigm]
Query: "What causes seasonal allergies?" ──(Semantic Mapping)──> Target: "Allergies_Pollen_Immune_Response"
│
(Leverages pre-trained knowledge of "Pollen" and "Allergies")
When a generative retriever decodes a semantic identifier, it isn't guessing a random number; it is expressing the semantic content of the document it wants to retrieve. If a query asks about "seasonal allergies," the model's attention heads naturally activate pathways related to "pollen," "histamines," and "immune responses." If the identifier for the relevant document contains those exact terms, the model can seamlessly transition from the query to the identifier using its pre-existing weights.
Zero-Shot Generalization
This alignment with pre-trained knowledge yields a massive practical advantage: zero-shot generalization.
In ZeroGR: A Generalizable and Scalable Framework for Zero-Shot Generative Retrieval, Weiwei Sun et al. demonstrated that using keyword-rich, semantic sentences as document identifiers allows generative retrieval models to generalize to entirely unseen tasks and domains without any task-specific training data. Because the identifiers share a common semantic space with the queries, the model can infer connections to new documents purely through linguistic analogy.
By shifting the architecture of document indexes from structural codes to natural language, this research direction transforms generative retrieval from a fragile exercise in database memorization into a robust, flexible extension of language understanding.
Key Foundational Work
Biomedical relation extraction via knowledge-enhanced reading comprehension (2022) — contributed 1 concept elements to this direction.
PROP: Pre-training with Representative Words Prediction for Ad-hoc Retrieval (2020) — contributed 1 concept elements to this direction.