Search engines often fail on expert questions because the question does not name the principle the answer depends on. A math question never says "pigeonhole principle," and a biology question about a symptom never names the pathway behind it, so the document that explains that principle shares no words with the query. Pseudo-relevance feedback is the classic repair. It is a trick where the system assumes the top few search results are relevant and adds their words to the query. In expert domains it backfires, because those top results are mostly wrong and their words pull the query further off course.
We propose to spend extra model computation at search time on a different question than "is this document relevant?". The model reads the top results, including the ones a relevance judge would discard, and proposes candidate principles: the theorem, mechanism, API, or rule those documents invoke. It then checks each candidate against the corpus before using it to expand the query. The bet is that near-miss documents often name the right principle even when they answer the wrong question. If so, relevance filtering throws away exactly the signal that closes the domain-knowledge gap. Because our method samples and checks more candidates as the budget grows, it also gives a compute-scaling curve we can compare head-to-head against simply re-ranking more deeply.
Core Claim
On reasoning-intensive retrieval, query expansion improves when it is grounded in feedback documents at the level of principles rather than passages. Concretely, an LLM proposes $K$ principle hypotheses from the top-$N$ first-pass results and verifies each one by how well the corpus responds to it. At matched token budgets, this beats three alternatives: ungrounded reasoning-based expansion, expansion fed by relevance-filtered documents, and spending the same budget on deeper re-ranking. Gains should show up mainly as recall of gold documents that were absent from the original candidate pool. Re-ranking cannot produce that kind of gain by construction.
Where This Sits
Several papers motivate the gap. BRIGHT builds 1,398 expert queries across 12 domains where relevant documents share little surface overlap with the query. BM25, the standard keyword-matching retriever, scores 14.5 nDCG@10 there. Asking an LLM to reason before searching helps by up to 12.2 points, but that reasoning never looks at the corpus. Retro* and Rank1 scale reasoning at the re-ranking stage. Neither can recover a document the first pass missed, and Rank1 reports "overthinking," meaning it rejects valid passages for failing strict surface criteria.
One paper constrains the design. Compute Allocation for Reasoning-Intensive Retrieval Agents finds that query expansion gains only +1.1 nDCG@10 from a stronger model, and that chain-of-thought "thinking" during expansion slightly hurts. Re-ranking, by contrast, gains +7.5 from a stronger model and about 21% from going from 10 to 100 candidates. Taken at face value, this argues against our proposal. We read it differently: their expansion was single-pass and never saw retrieved documents. So their result is evidence that ungrounded reasoning at expansion time is wasted. It is not evidence that grounded reasoning is wasted. Our experiments are designed so that this reading can be falsified.
TongSearch-QR trains small models with reinforcement learning to rewrite queries before retrieval. It reaches 27.9 nDCG@10 on BRIGHT with BM25 and 31.9 with the ReasonIR retriever. Again, it does not read corpus documents.
Other papers supply pieces of our method. MILL cross-filters LLM-generated passages against BM25 results to fix "mutual misalignment," using similarity thresholds on general benchmarks. MuGI samples several pseudo-documents and pools them. Both operate on whole passages and judge them by similarity. We extract a much smaller unit, a named principle, and verify it through the corpus's response rather than surface similarity. That verification step borrows the idea from environment-augmented generation that deterministic external feedback prunes bad reasoning branches, with the search index playing the role of the environment.
A further negative result needs addressing directly. TheoremQA found that appending theorem definitions does not help models solve problems. We do not claim that naming a principle helps a model reason. We claim it helps a retriever find the document about that principle, which is a lexical bridging task. Still, it means hypothesis quality cannot be assumed, which is why every hypothesis is verified.
Research Program
The pipeline has four steps.
- First pass. Run BM25 for query $q$ and take the top-$N$ documents, with $N$ around 20.
- Propose principles. An LLM reads each document together with the query and writes short principle hypotheses: what concept would make a document relevant to this query, as suggested by this one. We sample until there are $K$ distinct hypotheses $c_1,\dots,c_K$ in total.
-
Verify each hypothesis. For each $c_i$, issue the expanded query $q \oplus c_i$ (the query with the principle appended), retrieve its top-$m$ documents $D_i$, and score them with a cheap pointwise judge $r(q,d)$: $$s(c_i) = \frac{1}{m}\sum_{d \in D_i} r(q,d) \;-\; \lambda \,\mathrm{overlap}(D_i, D_0).$$ Here $D_0$ is the original top-$m$ set, and the $\lambda$-weighted overlap term rewards hypotheses that bring new documents into view rather than restating the first pass.
-
Expand and finish. The top-scoring hypotheses are fused into the final expanded query, and the shared final re-ranker runs on its candidate pool.
Compute scales along $K$ and $m$, and we report every result against total LLM tokens consumed, including verifier calls.
The analysis tests the mechanism, not just the headline score. We annotate, on a sample of BRIGHT queries, whether the gold document's principle is explicitly named in documents the judge rated non-relevant. This gives a direct estimate of how much signal relevance filtering discards. Hyperparameters ($N$, $K$, $m$, $\lambda$) are tuned by leave-one-domain-out on BRIGHT, since it has no training split. We use the BRIGHT codebase for evaluation and a single open model of about 7B parameters for every LLM role, so that no method gains from a stronger model.
Baselines
Naive reference. BM25 alone, which BRIGHT reports at 14.5 average nDCG@10. Alongside it, we run BM25 followed by the shared final re-ranker with no expansion. Every method feeds the same re-ranker, so this second baseline isolates what expansion adds.
Strongest re-ranking competitor. Re-ranking deeper with more reasoning samples, in the style of Retro*. It reports 36.6 nDCG@10 at 7B and 40.6 at 32B with 16 reasoning samples. This is the strongest competitor because Compute Allocation argues that compute belongs exactly here. We run it with our base model, sweeping candidate depth and samples per document until it consumes the same token budget as our method. Beating it means a higher nDCG@10 at equal tokens across a range of budgets, not at a single point.
Strongest query-side competitor. Ungrounded reasoning-based expansion. We use TongSearch-QR-7B where its released weights are available (27.9 nDCG@10 with BM25, from TongSearch-QR), plus a MuGI-style variant that samples $K$ pseudo-documents from our base model at matched tokens. Both feed the same re-ranker.
Key ablation. Relevance-filtered pseudo-relevance feedback. The judge scores the top-$N$ documents, the ones it rates relevant are kept, and classical term expansion is applied to them. This is also a MILL-style cross-filter run at matched compute. It shares every component with our method except the move from "keep relevant passages" to "extract and verify principles from all passages." Two further ablations isolate the remaining ingredients: hypotheses drawn only from rejected documents, and hypotheses used without verification.
Why not just…?
Why not just spend the budget on deeper re-ranking? This is the real threat. A cheap non-thinking re-ranker reached 95.6% of the strongest model's performance at roughly 4x lower cost in Compute Allocation. Its weakness is recall: a document outside the BM25 pool can never be re-ranked into the top. We settle this with matched-token curves and by reporting recall@100 of the candidate pool before re-ranking. If deeper re-ranking matches us at every budget and our recall gains are negligible, the core claim fails.
Why not just filter the feedback documents with the re-ranker and run ordinary feedback? This is simpler, and plausibly strong. It is exactly our key ablation. Our hypothesis predicts it underperforms on the BRIGHT Theorems and Coding subsets, where near-misses name the right principle. It may tie on StackExchange domains. The rejected-documents-only ablation decides the matter: if hypotheses drawn only from rejected documents add nothing, the "near-miss signal" story is wrong and the simpler filter should be preferred.
Why not just let a trained model reason about the query without reading the corpus? TongSearch-QR shows this is cheap and strong. But for principles the model does not know, it can only guess. TheoremQA shows open models below 15% on such material, which suggests their guesses are often wrong. We expect our advantage to concentrate on queries where ungrounded rewrites name a wrong principle, and we will report the per-query overlap between the two methods' wins. The two are also compatible: our verifier can score TongSearch-QR's rewrites as additional hypotheses, so the comparison need not be a winner-take-all choice.
Proposed Experiments
The main evaluation is all 12 BRIGHT domains, reporting nDCG@10 and pre-re-ranking recall@100, as matched-token scaling curves against every baseline above. A secondary check uses BEIR's TREC-COVID and SciFact. These are domains with more lexical overlap, where MILL reports TREC-COVID nDCG@1000 rising from 42.04 to 52.53 (MILL). There we test that principle feedback does no harm when the vocabulary gap is small. No reported result was found on Lacuna for R2MED, so we use it only as an exploratory biomedical test.
Three outcomes would be decisive. First, our curve dominating the deeper re-ranking curve on the Theorems and Coding subsets. Second, a positive gain from the rejected-documents-only ablation. Third, a recall@100 increase that accounts for most of the nDCG gain.
If expansion with grounding still shows the flat returns that Compute Allocation found for ungrounded expansion, we will report that as confirming their conclusion. It would mean compute belongs at re-ranking even when expansion reads the corpus.
References
- BRIGHT: A Realistic and Challenging Benchmark for Reasoning-Intensive Retrieval
- Retro*: Optimizing LLMs for Reasoning-Intensive Document Retrieval
- Rank1: Test-Time Compute for Reranking in Information Retrieval
- Compute Allocation for Reasoning-Intensive Retrieval Agents
- TongSearch-QR: Reinforced Query Reasoning for Retrieval
- MILL: Mutual Verification with Large Language Models for Zero-Shot Query Expansion
- Exploring the Best Practices of Query Expansion with Large Language Models
- a1: Steep Test-time Scaling Law via Environment Augmented Generation
- TheoremQA: A Theorem-driven Question Answering Dataset