An Efficient Rubric-based Generative Verifier for Search-Augmented LLMs

Linyue Ma, Yilong Xu, Xiang Long, Zhi Zheng

Introduction

Search-augmented Large Language Models (LLMs), such as those employed in Retrieval-Augmented Generation (RAG) or agentic search systems, increasingly rely on Reinforcement Learning (RL) to optimize reasoning and retrieval strategies. A critical component of this optimization is the reward signal, which guides the model toward desirable outputs. However, existing reward mechanisms present a significant trade-off between verifiability and robustness. Rule-based rewards, such as Exact Match (EM), are objective but fragile, often failing due to minor paraphrasing or in long-form contexts. Conversely, generative rewards (LLM-as-a-judge) are robust but computationally expensive, slow, and often lack a factual, objective basis.

Comparison of reward mechanisms across short-form and long-form workloads.

To bridge this gap, we introduce Search-Gen-V, a 4B-parameter generative verifier that utilizes a nugget-as-rubric paradigm. This approach decomposes complex answers into atomic "nuggets" of information, providing a unified, verifiable, and efficient reward signal for both short-form and long-form search tasks.

Methodology

The core of Search-Gen-V is the transformation of evaluation from subjective preference to objective fact-checking against a structured rubric.

Automatic Rubric Construction

Scaling long-form evaluation requires an automated pipeline for rubric generation. The system employs an iterative mining process where it retrieves initial passages and uses an LLM to rewrite queries (e.g., via entity substitution) to uncover additional relevant data. From these passages, an LLM judge extracts semantically complete factual statements, or "nuggets." These nuggets are then filtered for quality, deduplicated, and weighted based on their importance (e.g., "vital" vs. "optional").

Search-Gen-V Model Architecture

The verifier is a specialized 4B-parameter model based on Qwen3-4B. Its training involves two primary stages:

  1. Distillation (SFT): The model is trained to mimic a high-capacity "teacher" (Gemini-2.5-Flash) across various output formats to ensure structural robustness.
  2. Reinforcement Learning (RL): The model is further refined using Decoupled Clip and Dynamic Sampling Policy Optimization (DAPO), optimizing for accuracy against the teacher, adherence to reasoning formats, and output consistency.

Experimental Setup

The system was evaluated across a diverse set of benchmarks:

  • Short-form tasks: HotpotQA, SimpleQA, and BrowseComp.
  • Long-form tasks: DeepResearch Bench and TREC datasets (TREC RAG 24, TREC DL).
  • Training Data: MS MARCO V2.1 and Researchy Questions were used for large-scale web content grounding.

Results and Discussion

Search-Gen-V demonstrates that specialized small models can match the performance of general-purpose models orders of magnitude larger.

F1 score vs. model parameters, highlighting Search-Gen-V-4B's efficiency.

Performance and Efficiency

Search-Gen-V-4B achieved an average F1 score of 0.72 on the validation set, nearly reaching the 0.74 F1 of its 235B-parameter teacher model. This represents a significant improvement over the base 4B model's score of 0.58. Furthermore, the model effectively addresses the "false negative" problem inherent in rule-based systems; in HotpotQA, Search-Gen-V correctly identified over 60% of samples that Exact Match had erroneously marked as incorrect.

Alignment with Human Metrics

On the DeepResearch Bench, Search-Gen-V's evaluations showed a 0.7 Pearson correlation with the "Comprehensiveness" metric from Gemini-2.5-Pro. This suggests that the nugget-based approach aligns well with the evaluative standards of state-of-the-art large models while remaining computationally efficient.

Conclusion and Future Work

Search-Gen-V provides a robust, verifiable, and efficient solution for rewarding search-augmented LLMs. By unifying short-form and long-form evaluation under the nugget-as-rubric paradigm, it enables high-quality RL training without the prohibitive costs of massive judge models. Future work will focus on reducing the latency of the offline rubric construction process and integrating the verifier into a live, closed-loop RL training environment.

Summary updated

Abstract

Search augmentation empowers Large Language Models with retrieval capabilities to overcome the limitations imposed by static parameters. Recently, Reinforcement Learning leverages tailored reward signals as a viable technique to enhance LLMs performing tasks involving search. However, existing reward modeling for search-augmented LLMs faces several limitations. Rule-based rewards, such as Exact Match, are verifiable but fragile to variations in expression and cannot be applied to long-form workloads. In contrast, generative rewards improve robustness, but designing verifiable and stable rewards for long-form workloads in dynamic corpora remains challenging and also incurs high computational costs. In this paper, we propose a unified and verifiable paradigm, ``nugget-as-rubric", which treats atomic information points as structured evaluation criteria for different search-augmentation workloads. Short-form tasks correspond to a single rubric, whereas long-form tasks expand to multiple rubrics aligned with the question’s information needs. To support long-form settings, we design an automatic rubric construction pipeline based on query rewriting, which can automatically retrieve passages relevant to each question and extract rubrics from them, both from static corpora and from dynamic online web content. Furthermore, we introduce \textbf{Search-Gen-V}, a 4B-parameter efficient generative verifier under our proposed verifiable paradigm, which is trained via the idea of distillation and a two-stage strategy. Experimental results show that Search-Gen-V achieves strong verification accuracy across different workloads, making it a scalable, robust, and efficient verifiable reward constructor for search-augmented LLMs.
BibTeX
@misc{ma2025efficient,
  author = {Linyue Ma and Yilong Xu and Xiang Long and Zhi Zheng},
  title = {{An Efficient Rubric-based Generative Verifier for Search-Augmented LLMs}},
  year = {2025},
  eprint = {2510.14660},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2510.14660},
}
Figure