Introduction
Search-augmented Large Language Models (LLMs), such as those employed in Retrieval-Augmented Generation (RAG) or agentic search systems, increasingly rely on Reinforcement Learning (RL) to optimize reasoning and retrieval strategies. A critical component of this optimization is the reward signal, which guides the model toward desirable outputs. However, existing reward mechanisms present a significant trade-off between verifiability and robustness. Rule-based rewards, such as Exact Match (EM), are objective but fragile, often failing due to minor paraphrasing or in long-form contexts. Conversely, generative rewards (LLM-as-a-judge) are robust but computationally expensive, slow, and often lack a factual, objective basis.
To bridge this gap, we introduce Search-Gen-V, a 4B-parameter generative verifier that utilizes a nugget-as-rubric paradigm. This approach decomposes complex answers into atomic "nuggets" of information, providing a unified, verifiable, and efficient reward signal for both short-form and long-form search tasks.
Methodology
The core of Search-Gen-V is the transformation of evaluation from subjective preference to objective fact-checking against a structured rubric.
Automatic Rubric Construction
Scaling long-form evaluation requires an automated pipeline for rubric generation. The system employs an iterative mining process where it retrieves initial passages and uses an LLM to rewrite queries (e.g., via entity substitution) to uncover additional relevant data. From these passages, an LLM judge extracts semantically complete factual statements, or "nuggets." These nuggets are then filtered for quality, deduplicated, and weighted based on their importance (e.g., "vital" vs. "optional").
Search-Gen-V Model Architecture
The verifier is a specialized 4B-parameter model based on Qwen3-4B. Its training involves two primary stages:
- Distillation (SFT): The model is trained to mimic a high-capacity "teacher" (Gemini-2.5-Flash) across various output formats to ensure structural robustness.
- Reinforcement Learning (RL): The model is further refined using Decoupled Clip and Dynamic Sampling Policy Optimization (DAPO), optimizing for accuracy against the teacher, adherence to reasoning formats, and output consistency.
Experimental Setup
The system was evaluated across a diverse set of benchmarks:
- Short-form tasks: HotpotQA, SimpleQA, and BrowseComp.
- Long-form tasks: DeepResearch Bench and TREC datasets (TREC RAG 24, TREC DL).
- Training Data: MS MARCO V2.1 and Researchy Questions were used for large-scale web content grounding.
Results and Discussion
Search-Gen-V demonstrates that specialized small models can match the performance of general-purpose models orders of magnitude larger.
Performance and Efficiency
Search-Gen-V-4B achieved an average F1 score of 0.72 on the validation set, nearly reaching the 0.74 F1 of its 235B-parameter teacher model. This represents a significant improvement over the base 4B model's score of 0.58. Furthermore, the model effectively addresses the "false negative" problem inherent in rule-based systems; in HotpotQA, Search-Gen-V correctly identified over 60% of samples that Exact Match had erroneously marked as incorrect.
Alignment with Human Metrics
On the DeepResearch Bench, Search-Gen-V's evaluations showed a 0.7 Pearson correlation with the "Comprehensiveness" metric from Gemini-2.5-Pro. This suggests that the nugget-based approach aligns well with the evaluative standards of state-of-the-art large models while remaining computationally efficient.
Conclusion and Future Work
Search-Gen-V provides a robust, verifiable, and efficient solution for rewarding search-augmented LLMs. By unifying short-form and long-form evaluation under the nugget-as-rubric paradigm, it enables high-quality RL training without the prohibitive costs of massive judge models. Future work will focus on reducing the latency of the offline rubric construction process and integrating the verifier into a live, closed-loop RL training environment.
Summary updated