From Relevance to Utility: Rubric Alignment in Yuanbao Search

Yu Lu, Lixin Zou, Yangqi Chen, Hui Xu, Qiaolin Xia, Hao Wang, Qianwen Ma, Reed Li, Shuang Zeng, Haijin Liang, Haibo ShiarXiv 2026 · arXiv

TL;DR

Traditional search engines rank documents for human eyes, but AI agents need something different: a compact, high-utility set of evidence for reasoning. This paper introduces a "rubric-aligned" framework that unifies the entire search pipeline—retrieval, ranking, and selection—under a single query-specific utility objective, resulting in an 8.52% boost in answer quality for Tencent's Yuanbao Search.

What Problem Does This Paper Address?

We are moving from an era of "browsing" a list of blue links to "agentic search," where an LLM synthesizes a single answer from multiple sources. In this new model, the goal isn't just to find relevant documents; it's to assemble a document set that maximizes the utility for the downstream LLM.

The authors identify three major gaps in current systems:

  1. Granularity Mismatch: Retrieval and ranking stages usually score documents individually (pointwise), but utility is often a property of the whole set (e.g., "do these three documents collectively cover all facets of the query?").
  2. Annotation Cost: It is prohibitively expensive to have strong LLMs evaluate millions of query-document pairs to create training data.
  3. Sparse Rewards: In the final selection stage, a single score for a whole set of documents doesn't provide enough "credit assignment" to tell the model which specific token or choice was good or bad.
Figure 1: (a) Traditional search pipelines score documents by individual relevance, optimized for human browsing over ranked lists. (b) Prior agentic search work optimizes doc- ument selection for LLM synthesis, yet upstream retrieval and ranking remain unaligned to answer quality. (c) Our proposed framework unifies the entire search pipeline un- der a query-specific rubric: the rubric provides per query– document supervision for retrieval and ranking, and set-level reward for document selection, aligning all stages toward high-quality answers within the same latency budget.

Methods

The core innovation is the Query-Specific Rubric. For every query, the system generates a set of weighted positive criteria (what the evidence must have) and negative criteria (what it must avoid).

The framework operates in three main phases:

  1. Coarse-to-Fine Rubric Generation: Multiple strong LLMs (like ChatGPT and Claude) generate candidate criteria, which are then refined and anchored to the actual top-retrieved documents to ensure they are specific and verifiable.
  2. Utility Labeling via Distillation: To solve the cost problem, the authors train a lightweight "Utility Labeler" (7B parameters). It distills the reasoning of a massive teacher model into per-document grades (0–3), allowing the system to scale rubric-grounded supervision to millions of pairs.
  3. The "Rubric-Aligned" Pipeline:
    • Retrieval: A dual-encoder is fine-tuned using the utility labels rather than just click-log relevance.
  4. Ranking: A multi-task ranker uses four scoring heads (utility, authority, freshness, quality) to reorder documents.
  5. Selection: A dual-branch Reinforcement Learning (RL) approach uses GRPO (Group Relative Policy Optimization) for set-level rewards and Best-Rollout Distillation to provide stable, token-level guidance.
Figure 2: Overall architecture of the rubric-guided agentic search framework. Offline (top) involves coarse-to-fine rubric generation with weighted positive and negative criteria, utility-labeler training by teacher distillation, rubric-aligned retrieval and multi-task ranking, and dual-branch selection with GRPO and best-rollout distillation (OPD). Online (bottom) involves dense retrieval of candidate documents with the trained models, rubric-aligned re-ranking, set selection, and agent answer synthesis.

Datasets

The models were trained and evaluated on massive real-world datasets from Yuanbao Search:

  • Training: A corpus of 62 million queries and 610 million query-document pairs.
  • Evaluation Benchmarks: Four held-out sets: Random-1K (general), Long-Tail (rare queries), Entity (names/locations), and English.
  • Human Evaluation: A blind side-by-side (GSB) test on 1,000 sampled production queries.

Key Insights and Ideas

  • Rubrics as a Shared Surface: By using the same rubric to supervise retrieval, ranking, and selection, the authors ensure the "full stack" is aligned. In traditional systems, the retriever might find "relevant" documents that the selector later finds "useless" for reasoning.
  • Pointwise Amortization: The "Utility Labeler" effectively "amortizes" complex, set-level reasoning into simple pointwise scores. This allows the system to keep the efficiency of traditional retrieval while gaining the intelligence of LLM evaluation.
  • Dual-Branch RL: The combination of GRPO and Best-Rollout Distillation (OPD) is a clever fix for the sparse reward problem. GRPO handles the group-level comparison, while OPD "densifies" the signal by making the model imitate its own best performance.

Key Results

The rubric-aligned pipeline was deployed in production at Yuanbao Search, showing significant improvements:

  • End-to-End Quality: An 8.52% lift in Good-Same-Bad (GSB) scores over the production baseline.
  • Retrieval Recall: Utility-aligned fine-tuning improved Recall@1K by over 11 points on challenging "Entity" and "Long-Tail" queries.
  • Ranking Accuracy: The multi-task approach with focal loss significantly boosted the Positive-Negative Ratio (PNR), a measure of ranking alignment.
Figure 4: Training strategy ablation on the ranking bench- mark. Each bar adds the indicated strategy to Ranking-Base (PNR annotated at the bar end). ∗: significant over Ranking- Base (paired 𝑡-test, 𝑝< 0.05).

Limitations and Future Work

The authors note that while the system is robust, there are clear paths for expansion:

  • Dynamic Refreshing: Currently, rubrics are generated based on initial retrieval. Future work could involve dynamically updating rubrics as new, fresh web evidence is discovered during the search process.
  • Multimodal Search: The framework is currently text-centric. The authors aim to extend these rubrics to handle images, videos, and structured data.
  • Latency: While the labeler is lightweight, generating query-specific rubrics adds an offline cost; optimizing this for real-time "zero-shot" rubric generation is a potential next step.

Summary updated

Abstract

Agentic search demands that a search engine move beyond per document relevance ranking to assemble document sets optimized for downstream LLM reasoning. Prior work addresses this either by improving individual pipeline stages in isolation or by applying RL based selection on top of relevance trained upstream modules, leaving the full stack unaligned to utility for reasoning. We propose rubric aligned search, a framework that unifies the traditional search pipeline under a single supervision surface. For each query, we derive a rubric from coarse to fine, specifying weighted positive criteria that the document set must satisfy and negative criteria that it must avoid. From this rubric we derive two training signals. First, a lightweight labeler distills rubric grounded judgments into per document utility labels, which supervise a contrastive dual encoder for retrieval and a multi task ranker trained with four rubric aligned scoring heads. Second, a rubric conditioned evaluator produces a set level reward that drives a dual branch RL selector, combining GRPO with best rollout distillation to resolve credit assignment in subset optimization. Deployed in Yuanbao Search, the pipeline improves end to end answer quality by 8.52% over the online production baseline.
BibTeX
@misc{lu2026from,
  author = {Yu Lu and Lixin Zou and Yangqi Chen and Hui Xu and Qiaolin Xia and Hao Wang and Qianwen Ma and Reed Li and Shuang Zeng and Haijin Liang and Haibo Shi},
  title = {{From Relevance to Utility: Rubric Alignment in Yuanbao Search}},
  year = {2026},
  eprint = {2611.22503},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.22503},
}
Figure