Abstract: We propose Referral-Augmented Retrieval (RAR), a simple technique that concatenates document indices with referrals: text from other documents that cite or link to the given document. We find that RAR provides significant performance gains for tasks across paper retrieval, entity retrieval, and open-domain question-answering in both zero-shot and in-domain (e.g., fine-tuned) settings. We examine how RAR provides especially strong improvements on more structured tasks, and can greatly outperform generative text expansion techniques such as DocT5Query and Query2Doc, with a 37% and 21% absolute improvement on ACL paper retrieval, respectively. We also compare three ways to aggregate referrals for RAR. Overall, we believe RAR can help revive and re-contextualize the classic information retrieval idea of using anchor texts to improve the representations of documents in a wide variety of corpuses in the age of neural retrieval.
The Lexical Gap in Zero-Shot Retrieval
In "zero-shot" information retrieval, a system must find relevant documents for queries it has never seen before, often in domains it wasn't trained on. Modern "dense" retrievers (which use neural embeddings) often struggle in these scenarios, sometimes performing worse than classic keyword-based algorithms like BM25.
The core issue is the lexical gap: a user’s query might use different terminology than the target document. While previous research attempted to bridge this gap by using AI models to "imagine" potential questions a document might answer (generative expansion), these AI-generated queries are often noisy or nonsensical. We argue that we don't need to hallucinate context; we can find high-quality, human-written context in the existing web of citations and hyperlinks.
Methodology: Referral-Augmented Retrieval (RAR)
We propose Referral-Augmented Retrieval (RAR). Instead of just indexing a document's title and abstract, RAR finds "referrals"—passages from other documents that point to the target document.
For example, if you are indexing a famous AI paper like BERT, RAR would append sentences from other researchers' papers that describe BERT. These external descriptions often act as "gold-standard" summaries or reformulations that are much closer to how a real user might search for the paper. Crucially, RAR is training-free. It doesn't require fine-tuning the underlying model; it simply changes what information is stored in the retrieval index.
We experimented with three ways to combine these referrals:
- Concatenation: Appending the referral text to the document (optimal for BM25).
- Mean Pooling: Averaging the vector representations of the document and its referrals (optimal for dense models).
- Shortest Path: A retrieval-time trick treating each referral as a separate entry in the index.
Evaluation and Key Results
We evaluated RAR across several diverse benchmarks, including scientific paper retrieval (ACL, ArXiv, RefSeer), entity retrieval (DBPedia), and open-domain QA (NaturalQuestions, HotpotQA).
Key Findings:
- Massive Gains in Zero-Shot: On the ACL dataset, adding RAR to the SimCSE model boosted performance (Recall@10) from 0.160 to 0.355—a more than 2x improvement. For BM25 on the same dataset, performance jumped from 0.265 to 0.505.
- Superiority over Generative AI: RAR outperformed popular generative techniques like DocT5Query and Query2Doc. On ACL paper retrieval, RAR provided a 37% absolute improvement over DocT5Query. Human-written citations are simply more accurate and diverse than current LLM-generated queries for specialized domains.
- Dynamic Knowledge Updating: Because RAR is training-free, a document's representation can be updated just by adding new citations to the index. Adding citations from 2019 improved the retrieval of papers written in 2018, effectively "re-contextualizing" old knowledge with new trends.
- In-Domain Benefits: Even for models already trained on scientific text (like Specter), RAR provided a significant boost, proving it adds value beyond what the model learns during pre-training.
Conclusion and Future Work
RAR effectively uses the "wisdom of the crowd" to improve search. When someone cites a paper, they provide a high-quality, alternative "view" of that paper’s content. While RAR relies on an existing graph of citations or hyperlinks—making it less applicable to brand-new documents—it provides a robust, training-free way to enhance retrieval in structured corpora.
Future work could involve "implicit" referrals—using AI to find documents that should link to each other even if they don't—or using RAR to help "steer" retrieval systems via human-in-the-loop corrections.
Summary updated