TL;DR
Traditional search engines rank documents for human eyes, but AI agents need something different: a compact, high-utility set of evidence for reasoning. This paper introduces a "rubric-aligned" framework that unifies the entire search pipeline—retrieval, ranking, and selection—under a single query-specific utility objective, resulting in an 8.52% boost in answer quality for Tencent's Yuanbao Search.
What Problem Does This Paper Address?
We are moving from an era of "browsing" a list of blue links to "agentic search," where an LLM synthesizes a single answer from multiple sources. In this new model, the goal isn't just to find relevant documents; it's to assemble a document set that maximizes the utility for the downstream LLM.
The authors identify three major gaps in current systems:
- Granularity Mismatch: Retrieval and ranking stages usually score documents individually (pointwise), but utility is often a property of the whole set (e.g., "do these three documents collectively cover all facets of the query?").
- Annotation Cost: It is prohibitively expensive to have strong LLMs evaluate millions of query-document pairs to create training data.
- Sparse Rewards: In the final selection stage, a single score for a whole set of documents doesn't provide enough "credit assignment" to tell the model which specific token or choice was good or bad.
Methods
The core innovation is the Query-Specific Rubric. For every query, the system generates a set of weighted positive criteria (what the evidence must have) and negative criteria (what it must avoid).
The framework operates in three main phases:
- Coarse-to-Fine Rubric Generation: Multiple strong LLMs (like ChatGPT and Claude) generate candidate criteria, which are then refined and anchored to the actual top-retrieved documents to ensure they are specific and verifiable.
- Utility Labeling via Distillation: To solve the cost problem, the authors train a lightweight "Utility Labeler" (7B parameters). It distills the reasoning of a massive teacher model into per-document grades (0–3), allowing the system to scale rubric-grounded supervision to millions of pairs.
- The "Rubric-Aligned" Pipeline:
- Retrieval: A dual-encoder is fine-tuned using the utility labels rather than just click-log relevance.
- Ranking: A multi-task ranker uses four scoring heads (utility, authority, freshness, quality) to reorder documents.
- Selection: A dual-branch Reinforcement Learning (RL) approach uses GRPO (Group Relative Policy Optimization) for set-level rewards and Best-Rollout Distillation to provide stable, token-level guidance.
Datasets
The models were trained and evaluated on massive real-world datasets from Yuanbao Search:
- Training: A corpus of 62 million queries and 610 million query-document pairs.
- Evaluation Benchmarks: Four held-out sets: Random-1K (general), Long-Tail (rare queries), Entity (names/locations), and English.
- Human Evaluation: A blind side-by-side (GSB) test on 1,000 sampled production queries.
Key Insights and Ideas
- Rubrics as a Shared Surface: By using the same rubric to supervise retrieval, ranking, and selection, the authors ensure the "full stack" is aligned. In traditional systems, the retriever might find "relevant" documents that the selector later finds "useless" for reasoning.
- Pointwise Amortization: The "Utility Labeler" effectively "amortizes" complex, set-level reasoning into simple pointwise scores. This allows the system to keep the efficiency of traditional retrieval while gaining the intelligence of LLM evaluation.
- Dual-Branch RL: The combination of GRPO and Best-Rollout Distillation (OPD) is a clever fix for the sparse reward problem. GRPO handles the group-level comparison, while OPD "densifies" the signal by making the model imitate its own best performance.
Key Results
The rubric-aligned pipeline was deployed in production at Yuanbao Search, showing significant improvements:
- End-to-End Quality: An 8.52% lift in Good-Same-Bad (GSB) scores over the production baseline.
- Retrieval Recall: Utility-aligned fine-tuning improved Recall@1K by over 11 points on challenging "Entity" and "Long-Tail" queries.
- Ranking Accuracy: The multi-task approach with focal loss significantly boosted the Positive-Negative Ratio (PNR), a measure of ranking alignment.
Limitations and Future Work
The authors note that while the system is robust, there are clear paths for expansion:
- Dynamic Refreshing: Currently, rubrics are generated based on initial retrieval. Future work could involve dynamically updating rubrics as new, fresh web evidence is discovered during the search process.
- Multimodal Search: The framework is currently text-centric. The authors aim to extend these rubrics to handle images, videos, and structured data.
- Latency: While the labeler is lightweight, generating query-specific rubrics adds an offline cost; optimizing this for real-time "zero-shot" rubric generation is a potential next step.
Summary updated