Document Ranking via Query Likelihood Modeling

107 concepts84 papers1 proposal

In traditional information retrieval, we evaluate relevance by measuring the similarity between a query and a document. Whether using classical vector space models or modern neural cross-encoders that rely on a single classification token (such as BERT's [CLS]), the core question remains: How well does this document match the query?

Query likelihood modeling flips this question entirely. Instead of treating relevance as a matching score, it asks a generative question: If a user were thinking about the concepts in this document, how likely is it that they would generate this specific query?

The Intuition of Asymmetric Directionality

Why should we prefer calculating the probability of the query given the document ($P(Q|D)$) over the inverse ($P(D|Q)$)? The answer lies in the fundamental asymmetry between queries and documents.

Queries are concise, focused expressions of intent. Documents, on the other hand, are long, verbose, and filled with extraneous information. As Devendra Singh Sachan et al. point out in their work on the Unsupervised Passage Re-ranker (UPR) in Improving Passage Retrieval with Zero-Shot Question Generation, modeling $P(D|Q)$ forces a model to predict a vast, noisy document from a short prompt, which introduces massive variance. Conversely, evaluating $P(Q|D)$ keeps the target sequence ($Q$) constant across all candidate documents.

This directional choice yields two major advantages:

  1. Robustness to Document Length: As shown by Deepa Paranjpe et al. in Passage Scoring for Question Answering via Bayesian Inference on Lexical Relations, calculating $P(Q|D)$ avoids penalizing long documents simply because they contain non-query text. It focuses strictly on whether the document contains sufficient evidence to prompt the query.
  2. Elimination of Surface Form Biases: Because the target sequence remains identical when evaluating different candidate documents, the scoring metric is highly stable and free from length-induced calibration errors, a property leveraged by Muhammad Khalifa et al. in Few-shot Reranking for Multi-hop QA via Language Model Prompting.

From Statistical Frameworks to Modern LLMs

The concept of treating documents as probabilistic generative sources is not new; it originated in classical statistical language modeling for IR. As summarized by ChengXiang Zhai in Statistical Language Models for Information Retrieval, statistical language models move away from ad-hoc heuristics to treat each document as a unique probability distribution over a vocabulary. Early systems calculated relevance using Kullback-Leibler (KL) divergence to measure the statistical distance between a query's language model and a document's language model, utilizing smoothing techniques like Dirichlet smoothing to handle vocabulary sparsity, as evaluated by Paul Ogilvie in Experiments Using the Lemur Toolkit.

With the advent of large language models (LLMs), this generative paradigm has found powerful new life. Instead of counting unigram frequencies, modern systems leverage the deep semantic understanding of pre-trained autoregressive or encoder-decoder architectures. Cícero Nogueira dos Santos et al. demonstrated in Beyond [CLS] through Ranking by Generation that large-scale generative models like GPT-2 and BART can outperform traditional discriminative models by directly computing the conditional log-likelihood of a query given a document text.

Remarkably, this approach works exceptionally well in zero-shot settings. Shengyao Zhuang et al. showed in Open-source Large Language Models are Strong Zero-shot Query Likelihood Models for Document Ranking that off-the-shelf, open-source LLMs (like LLaMA) serve as highly effective zero-shot query likelihood models. By prompting the model with a document and measuring the average log-likelihood of generating the query tokens, the system performs fine-grained cross-attention without requiring any task-specific fine-tuning.

The Generality Trap and How to Fix It

A naive implementation of query likelihood modeling suffers from a distinct flaw: language models naturally assign higher probabilities to generic, high-frequency phrases (e.g., "What is...", "How to..."). If a candidate document assigns a high probability to a query simply because the query contains common words, the ranking will favor popular phrasing over precise relevance.

To counter this, researchers employ mutual information and contrastive scoring. Early dialogue systems, such as the one proposed by Haizhou Zhao et al. in SG01 at the NTCIR-13 STC-2 Task, addressed this by ranking candidates based on the probability of the unique context given the response, filtering out generic phrases.

More recently, Wei Fang et al. introduced Joint Passage Re-ranking (JPR) in Joint Inference of Retrieval and Generation for Passage Re-ranking, which optimizes for the Pointwise Mutual Information (PMI) between the query and the passage:

$$\text{Score}(D) = \log P(Q|D) - \log P(Q)$$

By subtracting the background probability of the query ($\log P(Q)$), the system penalizes generic queries and isolates the true, specific semantic connection between the candidate passage and the user's request. This ensures that the final ranking reflects true informational relevance rather than the statistical ubiquity of the query's surface form.

Key Foundational Work

Cross-lingual Information Retrieval Using Hidden Markov Models (2000) — contributed 1 concept elements to this direction.
History & citationVersion 2 · 2026-10-04

Current · Permanent link

  1. Version 2 (latest) · 2026-10-04 · 85 papers
  2. Version 1 · 2026-10-04 · 85 papers
Cite this direction

Latest version (a living page):

@misc{lacuna_generative_relevance_scoring_913a8c0e,
  title = {generative relevance scoring},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/generative-relevance-scoring/rdr_913a8c0e60a44149bbbe7b12fbfb315e},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 2, frozen text):

@misc{lacuna_generative_relevance_scoring_913a8c0e_v2,
  title = {generative relevance scoring},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/generative-relevance-scoring/rdr_913a8c0e60a44149bbbe7b12fbfb315e/v/2},
  note = {Version 2, frozen 2026-10-04. Accessed 2026-11-20},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake