TL;DR
CURATOR is a fine-tuned decision model designed to efficiently select the most relevant "spans" of evidence for Retrieval-Augmented Generation (RAG). By training on variable-sized candidate sets, it overcomes the scaling limitations of standard joint decision models, delivering a 21.7x speedup over context-aware baselines while significantly boosting evidence recall and downstream answer quality.
What Problem Does This Paper Address?
Retrieval-Augmented Generation (RAG) is the standard way to ground LLMs in external facts. However, current RAG systems are often "noisy." When we retrieve a passage, we usually feed the entire text to the LLM, even if only one or two sentences are actually relevant. This leads to several issues:
- Context Waste: Irrelevant tokens consume the limited context window.
- Performance Degradation: LLMs can get "lost in the middle" or distracted by irrelevant details, leading to incorrect answers.
- Efficiency: Processing hundreds of irrelevant tokens is computationally expensive.
The challenge is selecting evidence at a fine-grained level (sentences or spans) without losing the big picture. A single sentence might look irrelevant in isolation but become crucial when paired with another sentence found elsewhere in the retrieved set. While "context-aware" scoring can solve this, doing it for every single span individually is prohibitively slow.
Methods
The authors introduce CURATOR, which leverages Joint Context-Aware Scoring. Instead of evaluating each span one by one, CURATOR uses a "multi-readout" decision model to score all candidate spans in a single forward pass.
Key technical components include:
- Decision Model Backbone: Built on
pplx-decider-flash-v1.2(a 4.5B parameter model based on Qwen3.5). Unlike generative models that output text, decision models output logits for predefined choices, making them much faster for ranking. - Variable-Scale Joint Supervision: The model is trained on candidate sets of varying sizes ($K \in \{8, 16, 32, 64, 128\}$). This forces the model to maintain accuracy even when it has to make many simultaneous decisions.
- Class-Balanced Loss: Since relevant evidence is often sparse (e.g., only 3 out of 128 spans might be "gold"), the authors use a masked, class-balanced cross-entropy loss to prevent the model from simply predicting "irrelevant" for everything.
- Parameter-Efficient Fine-Tuning: They use LoRA (Low-Rank Adaptation) to update only ~3.8 million parameters, keeping the training lightweight.
- General Task Replay: To ensure the model doesn't "forget" how to do basic reasoning or classification, they interleave general decision tasks (like BoolQ or MNLI) during fine-tuning.
Datasets
The authors utilized a diverse array of datasets for training and evaluation:
- Training: QAMPARI (multi-answer QA), HotpotQA, and 2WikiMultiHopQA (multi-hop reasoning).
- Evaluation (Generalization):
- FEVEROUS: Fact verification involving both text and tables.
- MultiHiertt: Complex numerical reasoning over financial reports.
- CUAD: Legal contract review.
- MuSiQue: Downstream RAG evaluation for multi-hop questions.
Key Insights and Ideas
- The Scaling Wall: A major "aha" moment was discovering that pre-trained joint decision models degrade significantly as the number of candidates ($K$) increases. Interestingly, this isn't just because the input text gets longer; it's specifically the act of making more simultaneous decisions that confuses the model.
- Joint > Independent: Context matters. CURATOR outperforms independent span scoring because it can "see" how different pieces of evidence fit together to answer a complex question.
- Efficiency is Free (Almost): By using shared inference, CURATOR achieves the accuracy of expensive context-aware methods at the computational cost of simple, context-blind ones.
Key Results
CURATOR demonstrates impressive gains across the board:
- Efficiency: It is 21.7x faster and uses 25.7x fewer tokens for scoring compared to separate context-aware baselines.
- Recall: On the QAMPARI dataset, CURATOR improved evidence recall by an average of 25.8% over the base joint decision model.
- Downstream RAG: When the context budget is tight (e.g., 128 tokens), CURATOR improves downstream answer quality by 17.4% on average. It allows a RAG system to perform nearly as well with a tiny, curated context as it would with a massive, unfiltered one.
Limitations and Future Work
While CURATOR is a significant step forward, the authors note a few areas for improvement:
- Context Window Limits: If the retrieved pool is massive (hundreds of passages), it still won't fit in a single model invocation. CURATOR currently handles this by grouping candidates, but this means spans in different groups don't "see" each other.
- Calibration: Scores across different groups might not be perfectly calibrated, which could affect the final ranking.
- Agentic Workflows: The authors suggest that CURATOR's ability to dynamically share a token budget makes it a natural fit for AI agents that need to manage a "working memory" of tool outputs, a direction they leave for future research.
Summary updated