Abstract
Reranking is fundamental to information retrieval and retrieval-augmented generation, with recent Large Language Models (LLMs) significantly advancing reranking quality. While recent advances with LLMs have significantly improved document reranking quality, current approaches primarily rely on large-scale LLMs (>7B parameters) through zero-shot prompting, presenting high computational costs. Small Language Models (SLMs) offer a promising alternative because of their efficiency, but our preliminary quantitative analysis reveals they struggle with understanding task prompts without fine-tuning. This limits their effectiveness for document reranking tasks. To address this issue, we introduce a novel two-stage training approach, ProRank, for SLM-based document reranking. First, we propose a prompt warmup stage using reinforcement learning GRPO to steer SLMs to understand task prompts and generate more accurate coarse-grained binary relevance scores for document reranking. Then, we continuously fine-tune the SLMs with a fine-grained score learning stage without introducing additional layers to further improve the reranking quality. Comprehensive experimental results demonstrate that the proposed ProRank consistently outperforms both the most advanced open-source and proprietary reranking models. Notably, our lightweight ProRank-0.5B model even surpasses the powerful 32B LLM reranking model on the BEIR benchmark, establishing that properly trained SLMs can achieve superior document reranking performance while maintaining computational efficiency.
Introduction
In the modern landscape of Information Retrieval (IR) and Retrieval-Augmented Generation (RAG), reranking serves as a critical refinement step. After an initial fast retriever (such as BM25) identifies a broad set of candidate documents, a reranker re-orders them to ensure the most relevant information is presented to the user or the downstream generation model.
While Large Language Models (LLMs) have demonstrated exceptional zero-shot reranking capabilities, their deployment is often hindered by significant computational overhead and latency. Small Language Models (SLMs) present a more efficient alternative; however, they frequently suffer from a "prompt alignment" problem. Without specific fine-tuning, SLMs struggle to interpret complex task instructions and often fail to adhere to required output formats, limiting their utility as zero-shot rerankers. ProRank addresses this gap by enabling SLMs to achieve high-performance reranking through a targeted two-stage training strategy.
The ProRank Framework
ProRank utilizes a two-stage training pipeline designed to maximize the potential of SLMs without increasing architectural complexity.
Stage 1: Prompt Warmup via Reinforcement Learning
Instead of traditional Supervised Fine-Tuning (SFT), ProRank employs Group Relative Policy Optimization (GRPO) to "warm up" the model. This stage focuses on steering the SLM to follow task prompts and output binary relevance scores (0 or 1). The training is guided by a dual-reward system: a format reward, which ensures the model outputs a valid binary token, and an accuracy reward, which penalizes incorrect relevance judgments. This approach effectively teaches the model the "rules of the game" for reranking.
Stage 2: Fine-grained Score Learning
Binary scores are insufficient for high-precision ranking, as they cannot distinguish between multiple documents labeled as relevant. To overcome this, ProRank transitions to a fine-grained scoring mechanism. By aggregating the logits (raw prediction probabilities) of the "0" and "1" tokens, the model generates a continuous relevance score. The SLM is then further fine-tuned using a Cross-Encoder approach with binary cross-entropy loss to sharpen these scores, allowing for more nuanced document ordering.
Experimental Evaluation
The ProRank models, based on the Qwen2.5 architecture, were trained on the bge-m3 dataset. Evaluation was conducted across a diverse set of benchmarks to test generalizability:
- BEIR Benchmark: 14 English datasets covering domains like science, medicine, and general web search.
- CMTEB: Chinese retrieval datasets (Covid and DuReader) to test cross-lingual performance.
- COIR: The COSQA dataset for code-specific retrieval tasks.
Key Results and Insights
ProRank demonstrates that training strategy can be more impactful than raw parameter count. The models consistently outperformed significantly larger open-source models and high-end proprietary solutions.
Performance Against Giants
One of the most significant findings is that the ProRank-0.5B model achieved an average NDCG@10 of 55.57 on the BEIR benchmark. This performance surpasses the much larger Rank1-32B model, challenging the assumption that massive scale is a prerequisite for high-quality reranking. Furthermore, ProRank models matched or exceeded the performance of top-tier proprietary rerankers like Cohere-rerank-3.5 and Voyage-rerank-2.
Scaling and Robustness
The authors observed a "scaling paradox" in coarse-grained models, where performance sometimes dipped when moving from 0.5B to 1.5B parameters. However, the fine-grained scoring approach proved robust, showing consistent performance gains with increased model size. Additionally, the models exhibited strong zero-shot transfer capabilities, performing well in Chinese and Code domains despite being trained primarily on English data.
Conclusion and Future Work
ProRank establishes a new efficiency frontier for document reranking. By combining reinforcement learning for prompt alignment with logit-based fine-grained scoring, SLMs can achieve state-of-the-art results at a fraction of the computational cost of traditional LLMs.
Future research will focus on addressing the "noise ceiling," where performance can degrade when reranking very large sets of documents (e.g., top-5,000). Potential solutions include adaptive top-k selection to dynamically determine the optimal number of documents to rerank for a given query, further enhancing the robustness of SLM-based retrieval systems.
Summary updated