Synthetic Query Generation for Neural Retrieval

This direction is merged into another direction: LLM-based data augmentation. Its last version stays readable and citable here.

Version 1 of 133 papers34 conceptsfrozen 2026-10-04

Modern information retrieval has shifted from simple keyword matching to dense neural retrieval, where models like BERT encode queries and documents into a shared vector space. However, these neural models are notoriously "data-hungry." While they perform exceptionally well on general-domain datasets like Wikipedia, they often fail in specialized fields—such as medicine, finance, or law—where human-annotated query-document pairs are scarce or non-existent.

To bridge this "generalization gap," researchers have converged on a powerful paradigm: Synthetic Query Generation. Instead of waiting for users to ask questions, we use Large Language Models (LLMs) to "reverse-engineer" potential queries from a collection of unlabeled documents.

The intuition behind synthetic query generation is simple: if a document contains an answer, a sufficiently capable language model can imagine the question.

In a typical pipeline, such as the one described in Large Language Models for Information Retrieval: A Survey by Yutao Zhu, an LLM acts as an automated labeler. By prompting the model with a document (or a snippet of it), researchers generate "hypothetical" queries that a user might search to find that specific information. This creates a massive, synthetic training corpus of (Query, Document) pairs.

This approach has proven transformative for niche domains. For example, Mano Vikash Janardhanan and colleagues developed BAM embeddings for the financial sector by generating over 14 million synthetic pairs. This allowed a specialized retriever to outperform general-purpose models like OpenAI’s embeddings by a massive margin in financial contexts.

Why It Works: LLMs as Proxy Users

Synthetic generation works because LLMs capture the semantic "style" and terminology of a domain better than a generic retriever can. When Dheeru Dua and Pat Verga explored domain adaptation in Open-Domain QA, they found that prompting an LLM with just a few "seed" examples allows it to generate domain-specific questions that reflect the nuances of technical forums or medical texts.

Essentially, the LLM serves as a proxy for a human expert. It understands that in a medical context, a user might ask about "myocardial infarction" rather than just "heart attack." By training a smaller, faster retriever (like a bi-encoder) on these synthetic queries, we "distill" the LLM's vast linguistic knowledge into an efficient search tool.

Refining the Signal: Contrast and Consistency

Generating queries from single documents can sometimes produce generic or noisy results. To solve this, researchers have introduced more sophisticated generation strategies:

  1. Contrastive Generation: In MetaAdaptRank, Si Sun and Chenyan Xiong suggest generating queries by looking at pairs of documents. By asking the model to generate a query that applies to Document A but not Document B, the resulting synthetic data becomes much more discriminative, helping the retriever learn subtle differences in relevance.
  2. Consistency Checking: The InPars-Light framework by Leonid Boytsov and Preksha Patel introduces a "round-trip" validation. After generating a query for a document, they use a separate model to check if the original document is actually a good answer for that new query. If the "consistency" is low, the pair is discarded, ensuring the training data remains high-quality.

The Hybrid Future: Mixing Synthetic and Human Data

While synthetic data is powerful, it isn't a perfect replacement for humans. Dhananjay Ashok and Jonathan May identified a performance cliff where replacing the final 10% of human data with synthetic data causes a sharp drop in accuracy.

The most effective systems, like the COUGH dataset for COVID-19 FAQs developed by Xinliang Frederick Zhang and Huan Sun, use a hybrid approach. They leverage massive synthetic generation to provide a broad "understanding" of the domain and then fine-tune with a small set of high-quality, human-annotated pairs to ground the model in real-world user behavior.

By treating LLMs as tireless generators of "what-if" scenarios, researchers can now build high-performance search engines for any domain, no matter how specialized, without needing millions of manual labels.

Papers in this version

History & citationVersion 1 · 2026-10-04

Merged into another direction · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_synthetic_query_generation_for_neural_re_5f7cb4a8,
  title = {Synthetic Query Generation for Neural Retrieval},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/synthetic-query-generation-for-neural-retrieval/rdr_5f7cb4a8241a4066846134627b221ed2},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_synthetic_query_generation_for_neural_re_5f7cb4a8_v1,
  title = {Synthetic Query Generation for Neural Retrieval},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/synthetic-query-generation-for-neural-retrieval/rdr_5f7cb4a8241a4066846134627b221ed2/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}