Abstract
Information retrieval in professional domains often involves "pay-per-document" data sources where every access incurs significant costs. Traditional query expansion (QE) methods either suffer from noise in pseudo-relevance feedback or hallucinations in generative LLM approaches. We propose ProQE (Progressive Query Expansion), a framework that iteratively refines search queries by using an LLM to judge the relevance of retrieved documents. ProQE achieves a 37% improvement in retrieval accuracy over standard baselines while maintaining cost-efficiency in expensive API environments.
Introduction
Vocabulary mismatch—where a user's query terms differ from the technical terminology in relevant documents—remains a fundamental challenge in Information Retrieval (IR). While Query Expansion (QE) techniques address this by adding related terms, existing solutions are often unsuitable for high-stakes, cost-constrained environments like legal (Westlaw, LexisNexis) or medical databases.
Pseudo-Relevance Feedback (PRF) often introduces noise if the initial top-k results are irrelevant. Conversely, LLM-based expansion methods like query2doc can hallucinate non-existent facts. Furthermore, most methods assume document retrieval is free, ignoring the per-document fees common in professional APIs. ProQE addresses these issues by introducing a "cost-aware" progressive loop.
Methodology: The ProQE Framework
ProQE is a plug-and-play module compatible with both sparse (keyword-based) and dense (embedding-based) retrieval systems. The core innovation is an iterative, progressive loop that "homes in" on the target information.
The process follows five key steps:
- Step-by-Step Retrieval: Instead of a bulk fetch, ProQE retrieves only the top-1 document.
- LLM as a Relevance Judge: An LLM (e.g., T5-XL) evaluates if the retrieved document is relevant to the original query.
- Weighted Expansion: If relevant, the LLM extracts expansion terms and increases their weights. If irrelevant, weights are decreased to steer the search away from that semantic neighborhood.
- Iteration: This cycle repeats for $n$ iterations (the authors found 5 to be the optimal number).
- Final Polish: A final "Chain-of-Thought" (CoT) prompt adds a final layer of context before the final retrieval.
Experimental Evaluation
The researchers evaluated ProQE across four major IR benchmarks: Natural Questions (NQ), Web Questions (WQ), and TREC Deep Learning 2019 and 2020. These datasets provide a rigorous test for high-precision retrieval across diverse query types.
Key Results and Insights
ProQE demonstrates superior performance across all tested benchmarks and retrieval types.
Accuracy and Cost-Effectiveness
ProQE outperformed non-expanded baselines by an average of 37% in Mean Reciprocal Rank (MRR) and Recall@1. Notably, it consistently beat state-of-the-art generative methods like query2doc and GRF. By improving Recall@1, ProQE ensures that the correct document is found earlier in the process, directly reducing the number of expensive document fetches required.
Grounding and Negative Feedback
Unlike purely generative methods, ProQE grounds its expansion in actual retrieved documents, mitigating hallucinations. A critical insight was the value of negative feedback: by explicitly penalizing terms from irrelevant documents, the system learns to avoid "noisy" areas of the document space.
Conclusion and Future Work
ProQE offers a robust solution for high-precision retrieval in cost-sensitive environments. While the iterative nature introduces some latency, making it less suitable for real-time consumer search, it is ideal for professional "deep dive" research. Future work will explore applying ProQE to "black-box" APIs where the underlying search logic is opaque to the user.
Summary updated