TL;DR
ADORE is a novel query expansion framework that introduces a closed-loop, retrieval-grounded feedback loop to LLM-driven search. By iteratively generating expansions, observing how the target database responds, and explicitly grading the relevance of retrieved documents, ADORE turns query reformulation into a highly effective, corrective optimization process.
What Problem Does This Paper Address?
Query expansion improves search engine performance by adding relevant context or synonyms to a user's initial search query. Recently, Large Language Models (LLMs) have been widely used to generate these expansions—often by creating "pseudo-documents" that mimic what a perfect answer might look like.
However, most existing LLM-based methods operate in a vacuum: they generate text based purely on the LLM's internal knowledge without checking how the actual database (corpus) responds. This lack of "retrieval-outcome awareness" leads to three major issues:
- Retrieval Drift: The LLM might generate a plausible-sounding expansion that inadvertently steers the search engine toward completely irrelevant documents.
- Shallow Corpus Signals: Existing iterative methods simply feed raw retrieved text back into the LLM prompt as context, expecting the model to magically figure out what worked and what didn't.
- Unverified Iteration: Without explicit evaluation at each step, multi-round expansion methods risk compounding errors and reinforcing misleading terms over time.
Methods
The authors introduce ADORE (ADapt, Observe, Relevance Evaluate), a framework that treats query expansion as an iterative retrieval refinement process.
As shown in Figure 1, each round consists of three distinct stages:
- Adapt: An LLM reformulator generates a set of pseudo-passages. In the first round, this is done zero-shot. In subsequent rounds, it is conditioned on a structured pool of feedback.
- Observe: The generated pseudo-passages are combined with the original query and sent to the retriever (e.g., BM25), which returns a ranked list of top documents.
- Evaluate: A separate LLM acts as a relevance assessor, grading the newly retrieved documents on a scale of 0 to 3 against the original query.
These grades partition the documents into structured feedback tiers: terms from highly relevant documents are marked to be reinforced, undercovered aspects are flagged to be incorporated, and misleading terms from false positives are explicitly marked to be suppressed.
To keep computational costs in check, ADORE utilizes Adaptive Termination. The loop stops early if all top documents achieve maximum relevance (quality saturation), if no new documents are surfaced (coverage saturation), or when it hits a maximum budget of $T=5$ rounds.
Datasets
ADORE was evaluated across three diverse benchmark families:
- Passage Retrieval: TREC Deep Learning tracks (DL19, DL20, and DL-Hard), built on the 8.8-million-passage MS MARCO collection.
- Cross-Domain Retrieval: Five distinct BEIR datasets spanning scientific fact-checking (SciFact), biomedical literature (TREC-COVID), finance (FiQA), entities (DBPedia), and news (TREC-NEWS).
- Reasoning-Intensive Retrieval: Seven sub-domains of the BRIGHT benchmark (including Biology, Robotics, and Stack Overflow), where queries require complex, multi-step reasoning rather than simple keyword matching.
Key Insights and Ideas
- Separating Generation from Evaluation: Rather than forcing a single LLM to act as both creator and critic simultaneously, ADORE splits the tasks. The relevance assessor provides clean, structured signals (what to keep vs. what to avoid), making subsequent generation rounds corrective rather than merely repetitive.
- Anchoring to the Original Query: By always evaluating retrieved documents against the user's original query (rather than the modified query), ADORE prevents the system from drifting off-topic across multiple rounds.
- Rapid Convergence: The authors found that ADORE captures the vast majority of its performance gains within the first 2 to 3 rounds, proving that adaptive early stopping is highly viable.
Key Results
ADORE consistently outperformed strong query expansion baselines across nearly all evaluation settings:
- BEIR Benchmark: ADORE improved average nDCG@10 by 24.5% over standard BM25 and by 3.6% over ThinkQE, the strongest prior iterative baseline. Notably, it elevated a simple sparse retriever (BM25) to outperform or match state-of-the-art dense retrievers like BGE-base without any model training.
- BRIGHT Benchmark: On reasoning-heavy tasks, ADORE achieved an average nDCG@10 of 0.379, representing a 122.9% improvement over BM25 and a 9.2% gain over the best baseline. It easily surpassed specialized reasoning models like ReasonIR-8B and heavy rerankers like Rank-R1-14B.
- Generalizability: While ADORE's queries were optimized using BM25, the resulting expansions successfully transferred to dense retrievers (like BGE and Contriever), boosting their performance as well.
Limitations and Future Work
The primary trade-off in ADORE is latency. Running up to 5 rounds of retrieval and LLM-based document assessment adds significant computational overhead, making it less suitable for real-time, latency-sensitive production environments in its current form.
Additionally, the framework relies heavily on the accuracy of the LLM judge, which may introduce systematic biases in highly specialized or underrepresented domains. Future work includes exploring ADORE's efficacy in multilingual settings and optimizing it for learned sparse retrieval frameworks.
Summary updated