CURATOR: Scalable Fine-grained Evidence Selection for Retrieval-Augmented Generation using Decision Models

Shib Dasgupta, Bandish Parikh, Rashmi Gangadharaiah

TL;DR

CURATOR is a fine-tuned decision model designed to efficiently select the most relevant "spans" of evidence for Retrieval-Augmented Generation (RAG). By training on variable-sized candidate sets, it overcomes the scaling limitations of standard joint decision models, delivering a 21.7x speedup over context-aware baselines while significantly boosting evidence recall and downstream answer quality.

What Problem Does This Paper Address?

Retrieval-Augmented Generation (RAG) is the standard way to ground LLMs in external facts. However, current RAG systems are often "noisy." When we retrieve a passage, we usually feed the entire text to the LLM, even if only one or two sentences are actually relevant. This leads to several issues:

  1. Context Waste: Irrelevant tokens consume the limited context window.
  2. Performance Degradation: LLMs can get "lost in the middle" or distracted by irrelevant details, leading to incorrect answers.
  3. Efficiency: Processing hundreds of irrelevant tokens is computationally expensive.

The challenge is selecting evidence at a fine-grained level (sentences or spans) without losing the big picture. A single sentence might look irrelevant in isolation but become crucial when paired with another sentence found elsewhere in the retrieved set. While "context-aware" scoring can solve this, doing it for every single span individually is prohibitively slow.

Figure 1: Motivation for fine-grained evidence selection and joint scoring. (a) Retrieved passages contain relevant evidence alongside unnecessary information. Selecting useful spans reduces the input context passed to the answering model, leaving more room for other information within a limited context budget. (b) Three strategies for scoring candidate spans. Independent scoring considers each span separately, without taking other candidates into account. Separate context-aware scoring accounts for all the spans but requires repeatedly processing the shared context. Joint context-aware scoring evaluates multiple candidates through shared inference, avoiding redundant computation over the full context.

Methods

The authors introduce CURATOR, which leverages Joint Context-Aware Scoring. Instead of evaluating each span one by one, CURATOR uses a "multi-readout" decision model to score all candidate spans in a single forward pass.

Key technical components include:

  • Decision Model Backbone: Built on pplx-decider-flash-v1.2 (a 4.5B parameter model based on Qwen3.5). Unlike generative models that output text, decision models output logits for predefined choices, making them much faster for ranking.
  • Variable-Scale Joint Supervision: The model is trained on candidate sets of varying sizes ($K \in \{8, 16, 32, 64, 128\}$). This forces the model to maintain accuracy even when it has to make many simultaneous decisions.
  • Class-Balanced Loss: Since relevant evidence is often sparse (e.g., only 3 out of 128 spans might be "gold"), the authors use a masked, class-balanced cross-entropy loss to prevent the model from simply predicting "irrelevant" for everything.
  • Parameter-Efficient Fine-Tuning: They use LoRA (Low-Rank Adaptation) to update only ~3.8 million parameters, keeping the training lightweight.
  • General Task Replay: To ensure the model doesn't "forget" how to do basic reasoning or classification, they interleave general decision tasks (like BoolQ or MNLI) during fine-tuning.

Datasets

The authors utilized a diverse array of datasets for training and evaluation:

  • Training: QAMPARI (multi-answer QA), HotpotQA, and 2WikiMultiHopQA (multi-hop reasoning).
  • Evaluation (Generalization):
    • FEVEROUS: Fact verification involving both text and tables.
  • MultiHiertt: Complex numerical reasoning over financial reports.
  • CUAD: Legal contract review.
  • MuSiQue: Downstream RAG evaluation for multi-hop questions.
Table 1: Evaluation dataset statistics. Candidate and gold counts are means per question. Density is the mean per-question fraction of candidate units contain- ing annotated evidence. Pool coverage is the mean fraction of annotated evidence available before selec- tion. FEVEROUS includes annotated evidence pages and eight BM25 distractor pages per claim; its full cov- erage is by construction.

Key Insights and Ideas

  • The Scaling Wall: A major "aha" moment was discovering that pre-trained joint decision models degrade significantly as the number of candidates ($K$) increases. Interestingly, this isn't just because the input text gets longer; it's specifically the act of making more simultaneous decisions that confuses the model.
Figure 2: Scaling limitations of pretrained joint de- cisions. Evidence-discrimination AUROC decreases as the number of simultaneous decisions increases on multi-answer question answering and fact verification tasks. Higher is better.
  • Joint > Independent: Context matters. CURATOR outperforms independent span scoring because it can "see" how different pieces of evidence fit together to answer a complex question.
  • Efficiency is Free (Almost): By using shared inference, CURATOR achieves the accuracy of expensive context-aware methods at the computational cost of simple, context-blind ones.

Key Results

CURATOR demonstrates impressive gains across the board:

  • Efficiency: It is 21.7x faster and uses 25.7x fewer tokens for scoring compared to separate context-aware baselines.
  • Recall: On the QAMPARI dataset, CURATOR improved evidence recall by an average of 25.8% over the base joint decision model.
  • Downstream RAG: When the context budget is tight (e.g., 128 tokens), CURATOR improves downstream answer quality by 17.4% on average. It allows a RAG system to perform nearly as well with a tiny, curated context as it would with a massive, unfiltered one.
Table 2: Evidence selection quality and efficiency on QAMPARI. Evidence recall is reported at three selected- context budgets. Scoring latency is averaged per question, while input tokens are median totals per question. Candidate scoring is performed once, independently of the selected-context budget. Results are from the development set, which overlaps with checkpoint-selection data.

Limitations and Future Work

While CURATOR is a significant step forward, the authors note a few areas for improvement:

  • Context Window Limits: If the retrieved pool is massive (hundreds of passages), it still won't fit in a single model invocation. CURATOR currently handles this by grouping candidates, but this means spans in different groups don't "see" each other.
  • Calibration: Scores across different groups might not be perfectly calibrated, which could affect the final ranking.
  • Agentic Workflows: The authors suggest that CURATOR's ability to dynamically share a token budget makes it a natural fit for AI agents that need to manage a "working memory" of tool outputs, a direction they leave for future research.

Summary updated

Abstract

Retrieval-augmented generation (RAG) provides entire retrieved passages to the LLM, even though only a handful of spans within them contain the evidence pertaining to the question. While precise selection of these spans can improve context utilization, identifying them in isolation is challenging. A span may appear irrelevant by itself, and deciding whether to retain it may require examining other retrieved information. One can evaluate each span's candidacy with the full retrieved context, but this is computationally inefficient as the same text must be processed repeatedly. Pretrained decision model can be used to efficiently evaluate multiple spans simultaneously, allowing each decision to consider the surrounding spans as well. However, our experiments show that their ability to identify relevant evidence deteriorates as the number of simultaneous decisions increases. To address this, we introduce CURATOR, a fine-tuned decision model trained to make reliable evidence-selection decisions over large candidate sets. By training a decision model to select varying numbers of supporting spans from large candidate pools, CURATOR develops the capability to select evidence for RAG while retaining the computational efficiency of joint decision-making. We observe that CURATOR maintains a strong evidence discrimination capability as the number of candidates increases, generalizes across different tasks and document domains, and improves downstream RAG performance, particularly when the available context is limited.
BibTeX
@misc{dasgupta2026curator,
  author = {Shib Dasgupta and Bandish Parikh and Rashmi Gangadharaiah},
  title = {{CURATOR: Scalable Fine-grained Evidence Selection for Retrieval-Augmented Generation using Decision Models}},
  year = {2026},
  eprint = {2611.16366},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.16366},
}
Figure