Abstract
Large Language Models (LLMs) have become foundational in information retrieval (IR), particularly for query expansion. This paper introduces Multi-Text Generation Integration (MuGI), a training-free framework that leverages LLMs to generate multiple pseudo-references. By integrating these references with user queries, MuGI enhances both sparse and dense retrievers. Our findings demonstrate that increasing LLM samples improves performance, and effective integration strategies are critical. Notably, MuGI enables a 23M-parameter model to outperform a 7B-parameter baseline, highlighting the power of contextual information in query representation.
Introduction: Solving the Vocabulary Mismatch
In Information Retrieval, the "vocabulary mismatch" problem occurs when a user's query terms do not align with the specific wording of relevant documents. While query expansion using LLMs (like the HyDE or query2doc methods) has shown promise, existing approaches often struggle to balance generated text with original intent, maintain compatibility across different retriever types (sparse vs. dense), and manage the strict token limits of neural models.
The MuGI Framework
MuGI addresses these challenges through a two-stage "Retrieve & Rerank" pipeline designed to be training-free and versatile.
- Multi-Text Generation: MuGI generates multiple (typically five) pseudo-passages using zero-shot prompting to capture a broader context.
- Adaptive Reweighting for Sparse Retrieval: To ensure the original query isn't overshadowed in BM25 searches, MuGI uses an adaptive factor ($\lambda$) to repeat the original query based on the length of the generated passages.
- Contextualized Pooling for Dense Retrieval: To bypass token limits, MuGI encodes each (Query + Passage) pair separately and averages the embeddings into a single contextualized vector.
- Neural Calibration: Inspired by the Rocchio algorithm, MuGI refines the query vector by shifting it toward generated passages and away from potential "hard negatives."
Key Insights and Results
Evaluation on TREC DL and BEIR benchmarks revealed several critical insights into the effectiveness of query expansion:
- Quantity and Quality: Increasing the number of LLM-generated samples consistently improves retrieval performance, though benefits typically plateau after five passages. Furthermore, the choice of LLM matters; ChatGPT-4 consistently provides superior expansions compared to smaller or older models.
- Small Models, Big Impact: A striking result is that a tiny 23M-parameter model (MiniLM) using MuGI can outperform a 7B-parameter baseline (E5-Mistral). This suggests that high-quality query representation can compensate for smaller model sizes.
- Broad Applicability: MuGI achieved significant gains across the board, improving BM25 performance by up to 19.8% on TREC DL and boosting various dense bi-encoders by over 7%.
Conclusion
MuGI demonstrates that multi-text integration is a powerful, versatile tool for modern IR systems. While the primary bottleneck remains the latency associated with generating multiple passages from high-end LLMs, the framework sets a new standard for query expansion quality. Future work will focus on optimizing this process, perhaps through distilled LLMs or more efficient Retrieval-Augmented Generation (RAG) architectures.
Summary updated