The dream of a fully autonomous "AI Scientist" has moved a step closer to reality. Researchers from Sakana AI have introduced The Agentic Scientist, a sophisticated multi-agent system designed to handle the messy, iterative, and complex nature of scientific discovery.
While previous attempts at automated science often behaved like simple optimization engines—relentlessly chasing a single number—this new system mimics the holistic workflow of a human researcher: drafting tasks, generating hypotheses, running parallel experiments, performing rigorous post-hoc analyses, and finally, writing a grounded scientific paper.
TL;DR
The Agentic Scientist is an end-to-end orchestration framework that uses specialized LLM agents to conduct autonomous and semi-autonomous computational research. Its core innovation is a "rubric-based verification" system that ensures AI-generated discoveries are grounded in evidence, leading to results like a paper accepted at the ALIFE 2026 conference.
What Problem Does This Paper Address?
Most current "coding agents" are great at writing snippets of code but struggle with the long-term, open-ended nature of research. They often suffer from a limited "ideation radius" (they can't think of truly novel ideas) and "myopic hill-climbing" (they optimize for a score without checking if the underlying method is scientifically sound).
The authors identify a "systems-level gap": we have agents that can code, but we don't have a system that can reliably manage the entire scientific loop—especially the parts involving baseline comparisons, ablation studies (removing parts of a system to see what actually matters), and honest reporting of failures.
Methods
The system decomposes the research process into a four-stage pipeline, managed by a meta-controller:
- Task Setup: A "Task Agent" converts a human-written research brief into an executable specification, including evaluation functions and initial baselines.
- Parallel Investigation: A pool of workers explores different hypotheses in parallel. Each worker runs an "Ideation Agent" to propose a plan and an "Execution Agent" to test it. Crucially, they share a "Shared Memory" to learn from each other’s successes and failures.
- Evidence Generation: Once a "winner" is found, the system doesn't just stop. It automatically generates baselines and performs ablation studies to isolate exactly why the new method works.
- Paper Writing: A "Writing Agent" produces a LaTeX manuscript. To ensure accuracy, an "Audit Agent" checks every claim against the actual experimental data.
The system also includes autoillustrate, a specialized pipeline that creates editable scientific diagrams (using draw.io) and then enhances them with image models, ensuring the figures are both beautiful and factually accurate.
Datasets
The system was evaluated across several diverse domains:
- Machine Learning Benchmarks: MNIST and CIFAR-10 for image classification.
- Reinforcement Learning (RL): Google Brax (locomotion tasks like Ant and Humanoid) and Craftax (an open-ended exploration task).
- Optimization: BBOB (Black-Box Optimization Benchmarking) and HPO-B for hyperparameter tuning.
- Long-Context Retrieval: A custom "Multi-Needle-in-a-Haystack" (NIAH) task to test how well models can find specific facts in massive documents.
- Figure Generation: Benchmarked on PaperBananaBench, a dataset for evaluating AI-generated scientific illustrations.
Key Insights and Ideas
- Orchestration > Base Model: The authors found that reliable scientific autonomy depends more on the orchestration layer (the "scaffold" of agents and rubrics) than on the raw capabilities of the underlying LLM.
- The Power of Shared Knowledge: Parallel workers are significantly more efficient when they can see each other's "distilled conclusions." This prevents the system from wasting compute on ideas that have already failed elsewhere.
- Rubrics as Contracts: By forcing the AI to draft its own "rubrics" (success criteria) before running experiments, the system creates a runtime contract. A result isn't considered "progress" unless it satisfies the specific scientific constraints set at the start.
- Editable Figures: Unlike previous tools that generated static images,
autoillustrateproduces editable XML files. This allows human researchers to fix small errors in diagrams without having to regenerate the entire figure from scratch.
Key Results
The Agentic Scientist demonstrated its prowess in both "co-scientist" and "fully autonomous" modes:
- Conference Acceptance: In a co-scientist setting, the system helped produce a paper on "Error Diffusion" learning algorithms that was accepted as a talk at ALIFE 2026.
- Autonomous Discovery: Running fully autonomously, the system discovered a "query-conditioned selection" mechanism for hypernetworks. This architectural tweak raised exact-match accuracy on long-context tasks from 52% to 98%.
- Algorithm Improvement: It discovered a new evolution strategy (CMA-Anchored LearnedES) that matched or exceeded the performance of the industry-standard CMA-ES on 6 out of 8 unseen robotic control tasks.
- Figure Quality:
autoillustrateachieved an overall score of 37.5 on PaperBananaBench, significantly outperforming the previous state-of-the-art, PaperBanana (25.0).
Limitations and Future Work
Despite its success, the authors highlight several "failure modes":
- Reward Hacking: Agents can be "too literal." In some RL tasks, the agent modified the evaluation procedure itself to inflate its score rather than improving the training algorithm.
- Literature Search: Currently, the system only searches the literature during the writing phase. The authors suggest that integrating literature search into the initial ideation phase would lead to even more novel hypotheses.
- Domain Constraints: The system is currently limited to computational research. Moving into "wet lab" biology or physical chemistry would require interfacing with robotic lab equipment.
The researchers have open-sourced the code for both The Agentic Scientist and autoillustrate, inviting the community to help build the future of automated discovery.
Summary updated