RaDeR: Reasoning-aware Dense Retrieval Models

Debrup Das, Sam O’Nuallain, Razieh RahimiEMNLP 2025

TL;DR

This paper introduces RaDeR, a novel approach to dense retrieval that explicitly incorporates reasoning capabilities. By training retrieval models on data synthesized from mathematical problem-solving trajectories generated by Large Language Models (LLMs), RaDeR significantly improves performance on reasoning-intensive tasks, outperforming existing methods and even surpassing BM25 on complex reasoning queries.

What Problem Does This Paper Address?

Traditional dense retrieval models excel at lexical and semantic matching but struggle when relevance prediction requires complex reasoning. This limitation hinders their effectiveness in augmenting LLMs for tasks that demand deeper understanding, such as mathematical problem-solving. Existing approaches often rely on LLMs to generate reasoning steps that are then used for retrieval, but these steps may not align with what's needed for effective retrieval. Furthermore, generating high-quality training data for reasoning-based retrieval, especially diverse and hard-negative samples, is a significant challenge.

Methods

RaDeR tackles this by developing reasoning-aware dense retrieval models. The core innovation lies in their data synthesis strategy, which leverages a retrieval-augmented Monte Carlo Tree Search (MCTS) framework to solve mathematical problems. This MCTS process explores reasoning trajectories, retrieving and applying theorems to intermediate subproblems. The key steps include:

  • Retrieval-Augmented Search-based Reasoning: An LLM uses MCTS to build a search tree of reasoning steps, augmented with retrieval of relevant theorems.
  • Self-Reflection and Summarization: Retrieved theorems are evaluated for relevance by the LLM, and irrelevant ones are used as hard negatives. Theorems are also summarized for better integration.
  • Data Synthesis: Reasoning trajectories from successful MCTS rollouts are used to generate diverse training samples, including:
  • LLM Reasoning Queries (qllmq): Queries generated by an LLM that reflect the information need of a subproblem, aiming for low lexical overlap with relevant theorems.
  • CoT Reasoning Queries (qCoT): Reasoning steps up to the query node are used as queries.
  • Question Queries (qquestion): The original math question is used as a query.
  • Lexical Queries (qlexical): Queries generated to have high term overlap with relevant theorems, ensuring performance on traditional retrieval tasks.

Datasets

  • Training Data Generation: The authors synthesize training data using mathematical problem-solving datasets:
  • MATH: A large dataset covering various math subjects and difficulty levels.
  • NuminaMath: A large-scale collection of competition-level math problems and solutions.
  • Evaluation Datasets:
    • BRIGHT: A benchmark for reasoning-intensive retrieval, including various splits like TheoremQA, Math, Coding, and StackExchange.
  • RAR-b: Another benchmark for reasoning-retrieval tasks, used for evaluating performance on Math and Coding splits.
  • MS MARCO: A widely used benchmark for traditional information retrieval (term matching).
  • TREC DL'19 & TREC DL'20: Datasets for traditional information retrieval.

Key Insights and Ideas

  • Reasoning is Crucial for Relevance: The paper strongly demonstrates that for many tasks, especially mathematical ones, reasoning is not just a step after retrieval but is integral to predicting relevance.
  • Synthesized Data is Powerful: RaDeR's success hinges on its novel data synthesis strategy. By generating diverse reasoning trajectories and carefully crafting queries (especially qllmq), they create highly effective training data for reasoning-based retrieval.
  • Generalization Beyond Math: RaDeR's models, trained primarily on mathematical reasoning data, show remarkable generalization to other reasoning tasks, including coding and general knowledge retrieval.
  • Efficiency Gains: RaDeR achieves competitive or superior performance while using significantly less training data compared to prior work (e.g., REASONIR).
  • First-Stage Reasoning Retrievers: RaDeR pioneers first-stage retrieval models that are inherently reasoning-aware, rather than relying solely on re-rankers.

Key Results

  • BRIGHT Benchmark: RaDeR models achieve state-of-the-art performance on BRIGHT, outperforming strong baselines by up to 2 points in nDCG. They show particular strength on Theorem-based and Coding splits.
  • Outperforming BM25: RaDeR is the first dense retriever to outperform BM25 when using Chain-of-Thought (CoT) reasoning steps as queries.
  • RAR-b Benchmark: RaDeR performs comparably to or better than strong open-source and proprietary models on the Math and Coding splits of RAR-b.
  • Data Efficiency: RaDeR uses only 2.5% of the training data used by REASONIR while achieving a 4.5% relative performance gain.
  • LLM Augmentation: Augmenting LLMs with RaDeR retrievers leads to significant improvements in QA accuracy, outperforming augmentation with other retrieval models.

Limitations and Future Work

  • Single Document Reasoning: Current RaDeR models primarily reason over single documents in isolation. Future work could explore joint reasoning over multiple documents for multi-hop QA tasks.
  • Lack of Explicit Explanations: RaDeR does not generate explicit explanations for its retrieval decisions. Developing reasoning-aware retrievers that provide interpretable explanations while maintaining efficiency is a future direction.
  • Noisy Training Data: The MCTS framework uses rewards based on the final answer. It's possible for incorrect CoT reasoning paths to lead to correct answers, potentially introducing noise into the training data.
  • Further Model Scaling: The paper notes consistent gains from model scaling, suggesting that larger models could yield even better performance.

Summary updated

BibTeX
@inproceedings{das2025rader,
  author = {Debrup Das and Sam O’Nuallain and Razieh Rahimi},
  title = {{RaDeR: Reasoning-aware Dense Retrieval Models}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2025},
}
Figure