Hypothetical Document Generation for Semantic Retrieval

You are reading version 1 of this direction, frozen 2026-10-04. The latest version is version 2.

Version 1 of 2277 papers362 conceptsfrozen 2026-10-04

Traditional information retrieval is often hamstrung by the "vocabulary mismatch" problem. A user might ask a short, informal question, while the relevant information is buried in a long, technical document. Even modern dense retrievers, which map text to high-dimensional vectors, struggle when the query is too brief to provide a clear semantic signal.

A powerful research direction has emerged to solve this by using Large Language Models (LLMs) not as final answer-generators, but as "pivots" in the retrieval process. By generating a hypothetical document or a pseudo-answer before searching, these systems bridge the gap between what a user asks and how information is actually stored.

Bridging the Semantic Gap

The core intuition behind this approach is that an LLM’s "hallucination" can be a feature rather than a bug. In Precise Zero-Shot Dense Retrieval without Relevance Labels, Luyu Gao and colleagues introduced Hypothetical Document Embeddings (HyDE). Instead of encoding a short query like "How do black holes form?" directly into a vector, the system first asks an LLM to write a fake, detailed paragraph answering that question.

Even if this synthetic paragraph contains factual errors, it captures the expected structure and vocabulary of a real scientific document. When this synthetic text is converted into a vector, it sits much closer to actual relevant documents in the embedding space than the original five-word query ever could. This technique allows for precise retrieval without requiring any human-labeled training data.

Expanding the Search Surface

While HyDE focuses on dense vector spaces, the same principle applies to traditional keyword-based search. Yuning Mao and Jiawei Han demonstrated this with Generation-Augmented Retrieval (GAR) in Generation-Augmented Retrieval for Open-Domain Question Answering.

GAR uses an LLM to "hallucinate" potential answers, related titles, or context clues, which are then appended to the original query. This process, often called Generative Query Expansion, provides a "semantic anchor." By adding these generated terms, a simple search engine like BM25 can find relevant documents that don't share any exact keywords with the original query but do match the LLM's predicted context. As noted in Query2doc: Query Expansion with Large Language Models, this expansion significantly boosts performance by providing the necessary vocabulary to bridge lexical gaps.

Hallucinating Structure and Intent

The "hypothetical" strategy isn't limited to prose. Researchers are applying it to structured data and specialized domains:

  • Database Retrieval: In CRUSH4SQL, Mayank Kothyari and team prompt an LLM to hallucinate a "minimal, ideal schema" for a natural language query. This synthetic schema acts as a bridge to retrieve the actual, complex database columns needed for Text-to-SQL tasks.
  • Knowledge Graphs: Xinke Jiang et al. developed HyKGE, which generates a preliminary hypothesis to extract entities for Knowledge Graph searches, helping the system understand vague medical queries (Think and Retrieval).
  • Tool Use: For AI agents, Yanfei Chen and Jinsung Yoon proposed Re-Invoke, which generates synthetic user queries to index technical tools, aligning developer-centric documentation with natural user language (Re-Invoke: Tool Invocation Rewriting for Zero-Shot Tool Retrieval).

Iterative Synergy

The most advanced versions of this idea move from a single "shot" of generation to a conversation between the generator and the retriever. Zhangyin Feng and Xiaocheng Feng explored this in Retrieval-Generation Synergy Augmented Large Language Models.

In these frameworks, such as ITER-RETGEN, the model generates a draft response, uses that draft to retrieve better documents, and then uses those documents to refine the response. This creates a feedback loop where the model's internal knowledge (the generation) and external evidence (the retrieval) continuously improve each other. Even an initially incorrect response provides semantic cues—like specific entities or keywords—that help the retriever narrow down the search space in the next iteration.

Why It Works: The Intuition of Alignment

The success of these methods across 200+ papers suggests a fundamental shift in how we view retrieval. Instead of trying to force a short query to match a long document, we use the LLM to transform the query into the shape of a document.

Whether it is generating "draft clues" from a compressed memory (MemoRAG) or creating "pseudo-queries" to serve as document identifiers (MINDER), the goal is the same: Semantic Space Alignment. By moving the search into a space where both the query and the target are represented as rich, descriptive text, we allow the retriever to see the underlying intent rather than just the surface-level words.

Papers in this version

Showing 60 of 277 papers.

History & citationVersion 2 · 2026-10-04

Current · Permanent link

  1. Version 2 (latest) · 2026-10-04 · 179 papers
  2. Version 1 · 2026-10-04 · 277 papers
Cite this direction

Latest version (a living page):

@misc{lacuna_generative_query_expansion_6699be4d,
  title = {generative query expansion},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/generative-query-expansion/rdr_6699be4da7a6465e8a4695b462a3ac32},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-21},
}

This exact version (version 1, frozen text):

@misc{lacuna_hypothetical_document_generation_for_sem_6699be4d_v1,
  title = {Hypothetical Document Generation for Semantic Retrieval},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/hypothetical-document-generation-for-semantic-retrieval/rdr_6699be4da7a6465e8a4695b462a3ac32/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-21},
}