Abstract
Learning to search is the task of building artificial agents that learn to autonomously use a search box to find information. So far, it has been shown that current language models can learn symbolic query reformulation policies, in combination with traditional term-based retrieval, but fall short of outperforming neural retrievers. We extend the previous learning to search setup to a hybrid environment, which accepts discrete query refinement operations, after a first-pass retrieval step via a dual encoder. Experiments on the BEIR task show that search agents, trained via behavioral cloning, outperform the underlying search system based on a combined dual encoder retriever and cross encoder reranker. Furthermore, we find that simple heuristic Hybrid Retrieval Environments (HRE) can improve baseline performance by several nDCG points. The search agent based on HRE (HARE) matches state-of-the-art performance, balanced in both zero-shot and in-domain evaluations, via interpretable actions, and at twice the speed.
Introduction
Modern Information Retrieval (IR) has increasingly relied on neural architectures, specifically dual encoders for semantic matching and cross-encoders for precise reranking. Despite their success, these models suffer from two primary limitations: zero-shot fragility and computational opacity. Neural models often struggle to generalize to out-of-domain datasets where traditional keyword-based methods like BM25 excel. Moreover, the state-of-the-art approach of reranking the top 1,000 documents with massive transformers is computationally expensive and lacks interpretability.
To address these issues, we introduce HARE (Hybrid Agent REtrieval), a search agent designed to "learn to search" by iteratively refining queries. By navigating a hybrid environment that combines dense and sparse retrieval, HARE achieves state-of-the-art performance with significantly improved efficiency and transparency.
Methodology
The Hybrid Retrieval Environment (HRE)
The foundation of our approach is the Hybrid Retrieval Environment (HRE), which integrates multiple retrieval strategies to leverage their complementary strengths. The environment consists of a dense dual encoder (GTR) for semantic matching, a sparse BM25 retriever for robust keyword search, and a T5-based reranker (T5-R) to aggregate results.
The HARE Agent
The agent, a T5-based Query Expander (T5-Q), operates in multi-step sessions. In each step, the agent observes the current query and the top-ranked documents to generate symbolic refinements. These refinements use operators such as +term (mandatory inclusion), -term (exclusion), or ^weight (boosting).
The agent is trained via behavioral cloning using a "Rocchio Session" oracle. This oracle identifies optimal keywords from the MS MARCO dataset to guide the agent's learning process.
Key Insights and Results
HARE demonstrates several advantages over traditional "black-box" rerankers:
- "Digging Deep": Unlike standard systems limited to the top 1,000 documents, HARE can uncover relevant documents originally ranked much lower by using targeted keywords to filter the search space.
- Symbolic Interpretability: The use of explicit operators makes the agent's search strategy transparent. For example, refining a query from "weather in Germany" to "+temperatures +average" clearly indicates the agent's intent.
- Efficiency: HARE achieves state-of-the-art performance (nDCG@10 of 0.521 on BEIR) while being twice as fast as MonoT5. It only requires reranking an average of 66.7 unique documents per query, compared to the 1,000 documents typically processed by cross-encoders.
- Zero-Shot Generalization: HARE significantly outperformed its base environment (HRE) by 1.4 points on the BEIR benchmark, proving that iterative searching adds substantial value in unseen domains.
Discussion and Future Work
While HARE shows great promise, there are areas for further development. The agent currently relies heavily on the + operator, suggesting that more complex operators or phrase matching could enhance its expressiveness. Additionally, the reranker's performance was found to decline when processing too many documents (k > 50), indicating a need for optimization to handle the "deep" documents unearthed by the agent. Future research will explore Joint Reinforcement Learning to further synchronize the agent and reranker for even higher performance.
Summary updated