RECON: Reasoning with Condensation for Efficient Retrieval-Augmented Generation

Zhichao Xu, Minheng Wang, Yawei Wang, Wenqian Ye, Yuntao Du, Yunpu Ma, Yijun Tian

Abstract

Retrieval-augmented generation (RAG) systems trained using reinforcement learning (RL) with reasoning are hampered by inefficient context management, where long, noisy retrieved documents increase costs and degrade performance. We introduce RECON (REasoning with CONdensation), a framework that integrates an explicit summarization module to compress evidence within the reasoning loop. Our summarizer is trained via a two-stage process: relevance pretraining on QA datasets, followed by multi-aspect distillation from proprietary LLMs to ensure factuality and clarity. Integrated into the Search-R1 pipeline, RECON reduces total context length by 35%, leading to improved training speed and inference latency, while simultaneously improving RAG performance on downstream QA benchmarks. Notably, it boosts the average EM score of the 3B model by 14.5% and the 7B model by 3.0%, showing particular strength in multi-hop QA. RECON demonstrates that learned context compression is essential for building practical, scalable, and performant RAG systems.

Introduction

Retrieval-Augmented Generation (RAG) has become the standard for grounding Large Language Models (LLMs) in external facts. Recently, researchers have started using Reinforcement Learning (RL) to train "agentic" RAG systems (like Search-R1) that can iteratively search, think, and reason. However, these systems face a major "context bloat" problem. Retrieved documents are often verbose, redundant, or filled with irrelevant noise. When these raw documents are shoved into the model’s context window, costs and latency spike, and the model's "cognitive efficiency" drops due to noise—a phenomenon often called "lost in the middle." RECON (REasoning with CONdensation) addresses this by integrating a dedicated summarization module directly into the reasoning loop.

Methods

RECON adds an explicit "condensation" step to the RAG pipeline. Instead of passing raw documents to the reasoning policy model, RECON passes them through a specialized summarizer first. The summarizer (based on Qwen2.5-3B) is trained in two stages:

  1. Relevance Pretraining: Using the MS MARCO dataset, the model learns to distinguish between useful and non-useful documents for a given query.
  2. Multi-aspect Distillation: The model is fine-tuned to mimic high-quality summaries from GPT-4o-mini, optimized across dimensions like clarity, completeness, and factual correctness.
The RECON architecture integrates a Summarizer within the Rollout Module to condense search results before they reach the Policy Model.

As the agent reasons, it retrieves documents, the summarizer condenses them into "evidence nuggets," and the policy model uses those nuggets to decide its next move or provide a final answer.

Results

The researchers evaluated RECON across seven diverse public benchmarks, including General QA (NQ, TriviaQA, PopQA) and Multi-Hop QA (HotpotQA, 2wikiMultihopQA, Musique, Bamboogle).

RECON significantly reduces normalized costs across context length, inference time, and turns used compared to the Search-R1 baseline.

RECON demonstrates that "condensation" is essential for practical RAG:

  • Context Reduction: It reduced the total context length by 35% on average.
  • Speed Gains: Inference latency dropped by 30.9%, and training speed improved by 5.2%.
  • Accuracy Boost: On multi-hop QA, RECON consistently outperformed the Search-R1 baseline. For the 3B model, performance increased by 14.5%, while the 7B model saw a 3.0% boost.
  • Fewer Turns: Higher quality evidence allowed the model to often require fewer search turns to find the correct answer.

Discussion

RECON proves that in RAG, "less is more." By removing distractions, the model actually reasons better. This is particularly evident in smaller models (3B), which are more sensitive to context noise; cleaning the input allows them to punch far above their weight class. The disentangled design also maintains a transparent reasoning chain, as humans can read the condensed evidence and see exactly what information the model used to reach its conclusion.

Future work includes addressing domain specificity (e.g., medicine or law) and reducing reliance on teacher models like GPT-4o-mini for distillation. Scaling this condensation approach to more complex, long-horizon agentic tasks remains a promising direction.

Summary updated

Abstract

Retrieval-augmented generation (RAG) systems trained using reinforcement learning (RL) with reasoning are hampered by inefficient context management, where long, noisy retrieved documents increase costs and degrade performance. We introduce RECON (REasoning with CONdensation), a framework that integrates an explicit summarization module to compress evidence within the reasoning loop. Our summarizer is trained via a two-stage process: relevance pretraining on QA datasets, followed by multi-aspect distillation from proprietary LLMs to ensure factuality and clarity. Integrated into the Search-R1 pipeline, RECON reduces total context length by 35\%, leading to improved training speed and inference latency, while simultaneously improving RAG performance on downstream QA benchmarks. Notably, it boosts the average EM score of the 3B model by 14.5\% and the 7B model by 3.0\%, showing particular strength in multi-hop QA. RECON demonstrates that learned context compression is essential for building practical, scalable, and performant RAG systems. Our code implementation is made available at https://github.com/allfornancy/RECON.
BibTeX
@misc{xu2025recon,
  author = {Zhichao Xu and Minheng Wang and Yawei Wang and Wenqian Ye and Yuntao Du and Yunpu Ma and Yijun Tian},
  title = {{RECON: Reasoning with Condensation for Efficient Retrieval-Augmented Generation}},
  year = {2025},
  eprint = {2510.10448},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2510.10448},
}
Figure