Introduction
Information retrieval (IR) has traditionally relied on the dual-encoder architecture, where queries and documents are embedded into a shared vector space for similarity matching. While effective, this paradigm is being challenged by Generative Retrieval (GR). In GR, a model—typically a sequence-to-sequence transformer like BART or T5—is trained to directly generate a unique identifier (ID) for a relevant document given a query. This approach treats retrieval as a differentiable sequence-to-sequence task, potentially capturing deeper semantic relationships than vector similarity alone.
However, the success of GR is heavily dependent on the choice of document identifiers. Existing methods often struggle with the "identifier problem": numeric IDs are difficult for models to memorize, titles are often missing or non-unique, and substrings may lack broader context. To address these limitations, researchers from The Hong Kong Polytechnic University and Microsoft introduced MINDER (Multiview Identifiers eNhanceD gEnerative Retrieval).
The Multiview Identifier Approach
MINDER posits that a single identifier type is insufficient to capture the multifaceted nature of a document. Instead, it represents each passage through three distinct "views":
- Titles: Leveraging existing metadata (e.g., Wikipedia headings) to provide a high-level summary.
- Substrings: Using actual spans of text from the document to ensure precision and grounding in the source content.
- Pseudo-queries (Synthetic Identifiers): Generating potential questions that the passage could answer. These act as a semantic bridge, aligning the document's representation with the natural language format of user queries.
By integrating these views, MINDER allows the model to learn multiple paths to the same document, increasing the likelihood of successful retrieval across diverse query types.
System Architecture and Implementation
The MINDER pipeline consists of three primary stages: training, constrained generation, and holistic ranking.
Training with Identifier Prefixes
MINDER utilizes a BART-large model trained to generate the multiview identifiers. To distinguish between views during training and inference, the model uses specific prefixes (e.g., title:, substring:, or pseudo-query:). This conditioning allows the model to learn the specific characteristics of each identifier type and leverage its pre-trained linguistic knowledge.
Constrained Generation via FM-index
To prevent the model from generating non-existent strings (hallucinations), MINDER employs constrained generation. It uses an FM-index—a compressed full-text substring index—to ensure that every generated token sequence corresponds to a valid identifier within the corpus.
Holistic Ranking and Aggregation
During inference, MINDER generates multiple candidate identifiers across all three views. Rather than relying on a single top-1 output, the system aggregates the scores from these multiple views to produce a final ranked list of passages. This redundancy ensures robustness; if a document lacks a clear title, it can still be retrieved via its substrings or synthetic queries.
Experimental Evaluation
The researchers evaluated MINDER on three benchmark datasets: Natural Questions (NQ), TriviaQA, and MSMARCO.
- Natural Questions (NQ): MINDER achieved a Hits@5 of 65.8, a significant improvement over the previous state-of-the-art GR model, SEAL, which scored 61.3.
- MSMARCO: This dataset is particularly challenging due to its scale and lack of structured titles. MINDER demonstrated that GR can scale to web-level tasks, achieving a Recall@100 of 78.7, compared to SEAL's 57.2.
- Ablation Insights: The study revealed that different views contribute differently depending on the task. Substrings were crucial for factoid-heavy datasets like NQ, while pseudo-queries were the most influential view for the noisier MSMARCO dataset.
Discussion and Limitations
While MINDER sets a new benchmark for generative retrieval, several challenges remain. The autoregressive nature of the model means it generates text linearly, which can lead to error propagation. Additionally, generating multiple identifiers per query increases inference latency (approximately 1.2x slower than the SEAL baseline). Finally, the use of an FM-index to store multiview identifiers increases the memory footprint, though this growth remains linear relative to the corpus size.
Conclusion
MINDER represents a significant step forward for generative retrieval by moving beyond single-identifier representations. By leveraging titles, substrings, and synthetic pseudo-queries, MINDER creates a more robust and semantically rich indexing mechanism. Its success on large-scale benchmarks like MSMARCO suggests that generative retrieval is a viable and powerful alternative to traditional dual-encoder systems.
Summary updated