Imagine All The Relevance: Scenario-Profiled Indexing with Knowledge Expansion for Dense Retrieval

Sangam Lee, Ryang Heo, SeongKu Kang, Dongha Lee

Abstract

Dense retrieval models often fail when a query requires deep reasoning or bridges different formats (like natural language to code). We present SPIKE (Scenario-Profiled Indexing with Knowledge Expansion), a framework that uses Large Language Models (LLMs) to "reframe" documents into multiple hypothetical retrieval scenarios. By indexing the reasoning process itself, SPIKE uncovers implicit relevance that surface-level embeddings miss. Our approach demonstrates significant performance gains across reasoning-intensive benchmarks, improving state-of-the-art retrievers by up to 20.7% in nDCG@10.

Introduction

Standard dense retrieval systems, such as DPR or BGE, typically represent documents as single vectors in a semantic space. While effective for keyword-adjacent or surface-level semantic matching, these systems struggle with reasoning-intensive retrieval. The authors identify two primary hurdles:

  1. Implicit Relevance: A query may logically relate to a document without direct semantic overlap (e.g., "Open Market Operations" vs. "Liquidity Coverage Ratios").
  2. Format Mismatch: Bridging the gap between natural language questions and documents containing code snippets or mathematical formulas is notoriously difficult for standard text-to-text models.

Methodology: The SPIKE Framework

SPIKE introduces a three-step framework to integrate reasoning directly into the search index.

The SPIKE framework: 1) Define Scenario with a high-performing LLM, 2) Distill into an efficient small LLM, and 3) Perform retrieval using scenario-augmented indices.

1. Scenario Generation

Using a high-performing teacher model (GPT-4o), the system analyzes each document to generate "scenarios." Each scenario consists of a Main Topic (M), Key Aspects (K), Information Needs (I), and an Explanation (E) that explicitly links the document's content to a hypothetical user need.

2. Knowledge Distillation

To manage computational costs, the reasoning capability is distilled into a smaller, efficient Scenario Generator (a fine-tuned Llama-3.2-3B). This allows for scalable processing of large document corpora.

3. Scenario-Profiled Indexing and Retrieval

During indexing, SPIKE stores vectors for both the document and the generated scenarios (specifically the combination of Main Topic + Explanation). At inference time, a weighted score is calculated: $$r_{final} = \alpha \cdot r_{doc} + (1 - \alpha) \cdot \max(r_{scenarios})$$ This dual-path approach ensures that a document is surfaced if either its content or its reasoning "scenarios" match the query.

Experimental Results

The framework was evaluated on the BRIGHT benchmark, designed for reasoning-intensive retrieval, and the BEIR benchmark for zero-shot generalization.

Performance comparison (nDCG@10) across various retrieval models with and without SPIKE on original and reasoning-heavy queries.

Key Findings

  • Significant Performance Gains: SPIKE consistently improved performance across all tested retrievers. On the BRIGHT benchmark, it boosted E5-Mistral by 20.7% and SFR-Embedding by 18.6% in nDCG@10.
  • The Importance of Explanations: Ablation studies showed that the "Explanation" (E) component is the most critical factor, acting as a bridge that translates document logic into formats matching complex user queries.
  • Zero-Shot Generalization: A scenario generator trained on the BEIR corpus (lacking code/math) still significantly improved retrieval on the code and math subsets of BRIGHT, indicating the framework learns general reasoning patterns.

Conclusion

SPIKE effectively addresses the limitations of dense retrieval in reasoning-heavy contexts by indexing hypothetical scenarios and their explanations. While it introduces some indexing overhead, it adds no latency at inference time and provides better contextual grounding for Retrieval-Augmented Generation (RAG) workflows. Future work will focus on scaling scenario-augmented training data to further improve out-of-domain generalization.

Summary updated

BibTeX
@inproceedings{lee2025imagine,
  author = {Sangam Lee and Ryang Heo and SeongKu Kang and Dongha Lee},
  title = {{Imagine All The Relevance: Scenario-Profiled Indexing with Knowledge Expansion for Dense Retrieval}},
  booktitle = {COLM},
  year = {2025},
  url = {https://openreview.net/forum?id=ZYVAtUUNbH},
}
Figure