TL;DR
LEAD-RL (Learning Effective Agent Delegation with Reinforcement Learning) introduces a "contribution-aware" credit assignment system for multi-agent research swarms. By using a training-time rubric to judge whether individual subagent reports were actually helpful, neutral, or harmful, the framework significantly improves both answer accuracy and computational efficiency without needing explicit rewards for speed or token usage.
What Problem Does This Paper Address?
Deep-research agents—AI systems designed to answer complex questions through iterative searching and tool use—face a major bottleneck: context bloat. As an agent searches, its "memory" (context window) fills up with search results, intermediate hypotheses, and failed attempts, making it harder to synthesize a final answer.
Delegation is a natural solution. A "main agent" can spin up "subagents" to investigate specific subtasks in isolated contexts, returning only a condensed report. However, training these swarms is tricky. Most current methods use outcome-only rewards: if the team gets the answer right, every agent gets a "good job" signal.
This creates a massive credit assignment problem. A subagent might provide a brilliant, helpful report, but if the main agent fumbles the final synthesis, the subagent is unfairly penalized. Conversely, a subagent might return a misleading "hallucination" that the main agent happens to catch and correct; in an outcome-only system, that harmful subagent would be rewarded for the team's success. This noise prevents swarms from learning how to coordinate efficiently.
Methods
The authors propose LEAD-RL, a joint training framework that treats delegation as a form of "active context management."
The core innovation is the Contribution-Aware Credit Assignment:
- Joint Training: Both the main agent and subagents are updated simultaneously from the same "swarm trajectories."
- The Rubric: During training, an external judge (like DeepSeek V4 Pro) evaluates each subagent's report against a reference answer and the specific subtask brief. It labels reports as Helpful (+1), Neutral (0), or Harmful (-1).
- Gated Advantage: This label acts as a gate for the team's reward.
- Helpful subagents only inherit positive team rewards (protecting them from team failures).
- Harmful subagents only inherit negative team rewards (preventing them from being rewarded for "lucky" team successes).
- Neutral subagents follow the standard team outcome.
Crucially, the final answer correctness remains the only objective. There are no hand-tuned rewards for using fewer tokens or being faster; efficiency is expected to emerge as a byproduct of better coordination.
Datasets
The researchers evaluated LEAD-RL across five diverse benchmarks to ensure the results weren't specific to one type of task:
- BrowseComp & BrowseComp-ZH: English and Chinese factual web search.
- GAIA: General assistant tasks requiring tool use.
- Humanity’s Last Exam (HLE): Expert-level academic questions.
- DeepSearchQA: Research tasks requiring set-valued answers (finding multiple items).
Key Insights and Ideas
- Delegation as Context Management: Instead of just "doing more work," delegation is framed as a way to control what information enters the main agent's global context. This makes the quality of the report the most important metric for a subagent.
- Efficiency via Reliability: The most surprising finding is that accuracy and efficiency are not necessarily a trade-off. By making subagent reports more reliable, the main agent performs fewer "compensatory" searches and delegates more selectively.
- Rubric-Guided Filtering: The rubric doesn't add a new "process reward." Instead, it acts as a noise filter for the existing outcome reward. This prevents the policy from drifting toward "busy work" that doesn't contribute to the final goal.
Key Results
LEAD-RL showed substantial improvements over standard joint training (where credit is shared equally):
- Accuracy vs. Efficiency: On the BrowseComp benchmark, LEAD-RL improved accuracy by 2.49 percentage points while simultaneously reducing the "critical-path cost" (a measure of execution depth) by 24.5%.
- Scaling Efficiency: A smaller model trained with LEAD-RL (124B parameters) outperformed a much larger model (DeepSeek V4 Flash, 284B parameters) while using fewer tokens and fewer critical steps.
- Validation Gains: During training validation, accuracy increased by 6.78 points while the computational cost remained nearly constant, proving that the model was learning to work smarter, not just harder.
Limitations and Future Work
While LEAD-RL improves the quality-efficiency frontier, the authors note a few areas for further exploration:
- Rubric Noise: The system relies on a high-quality judge model during training. If the judge is biased or noisy, it could suppress valid subagent behaviors.
- Fixed Weights: The relative weight of subagent learning is currently a hyperparameter that is annealed over time; finding a more dynamic way to balance main and subagent updates could yield better results.
- Mechanism vs. Hypothesis: While the authors hypothesize that the efficiency gains come from reduced "corrective" searching, they note that this mechanism hasn't been directly measured yet.
Overall, LEAD-RL demonstrates that in the world of multi-agent AI, better coordination—driven by smart credit assignment—is the key to breaking the "more agents = more cost" cycle.
Summary updated