TL;DR
DiscoveryAgent is a new framework for "scientist-centric" AI that balances autonomous research with human steering and rigorous auditing. By using nested loops for execution and a first-of-its-kind auditing system, it allows researchers to scale their work while maintaining full authority over the direction and integrity of their discoveries.
What Problem Does This Paper Address?
The recent surge in "AI Scientists" has created a polarizing design space. On one end, we have interactive coding assistants that require constant micro-management. On the other, we have end-to-end autonomous systems that ingest data and spit out a final report, leaving the human scientist as a passive observer of a "black box" process.
The authors argue that scientific progress isn't just about automating steps; it’s about deciding which observations are consequential and which assumptions should be challenged. Current autonomous systems often struggle with transparency—burying errors in massive logs—and lack "steerability," making it difficult for experts to redirect a failing investigation. DiscoveryAgent aims to bridge this gap by prioritizing steerability, transparency, and auditability.
Methods
DiscoveryAgent is built on a nested loop architecture:
- The Inner Loop (Execution): A "Director" agent coordinates specialized sub-agents for code execution, literature review, and web search to test a specific hypothesis.
- The Outer Loop (Steering): This is the boundary where the human scientist intervenes. Between research rounds, the scientist can inspect accumulated evidence and select, refine, or replace the next hypothesis.
Crucially, the system implements a rigorous checkpointing and provenance system. Every agent action, code change (via Git), and intermediate result is recorded in a structured tree. This enables Audit Agents—specialized AI tools that help scientists interrogate the research history, verify claims, and identify errors without manually reading thousands of lines of logs.
Datasets
The framework was validated across diverse domains using several key benchmarks and datasets:
- StarEmbed: ~40,000 expert-labeled light curves for variable-star classification.
- ProteinGym: 217 deep mutational scanning (DMS) assays used to predict how mutations affect protein function.
- DESI Data Release 1: Used for a case study in extragalactic astronomy.
- American Community Survey (ACS) & IPUMS: Used for an economics study on housing insurance.
- Perturb-seq atlas: Genomic data used to identify autoimmune risk genes.
Key Insights and Ideas
- The "Auditability" Multiplier: The most striking finding wasn't just that AI can find errors, but that it empowers humans to do so. Scientists collaborating with audit agents identified 14x more valid errors than those performing manual reviews of paper and code.
- Steering Beats Pure Autonomy: In the ProteinGym benchmark, the version of the agent steered by a human expert (DA-Steer) significantly outperformed the fully autonomous version (DA-Auto), ranking 2nd out of 113 total methods.
- Multimodal Discovery: In the astronomy case study, the agent discovered that combining image morphology (what a galaxy looks like) with photometry (its brightness) resolved complementary failure modes, leading to a highly accurate redshift estimator.
- Relay Genes: In functional genomics, the agent identified "relay genes"—genes that are both highly regulated and exert broad influence—as being significantly enriched for autoimmune disease associations, providing concrete targets for future lab work.
Key Results
- Astronomy: DiscoveryAgent built a variable-star classifier that achieved the highest balanced accuracy (86.23%) among all evaluated AI baselines. In a steered case study, it achieved a 0.971 AUROC for low-redshift galaxy classification.
- Biology: The steered agent surpassed 99% of the 106 ProteinGym baselines, including other popular AI scientist models.
- Economics: The agent documented a staggering 83% rise in Florida homeowners-insurance premiums from 2013 to 2023. It revealed a regressive burden: the lowest income quartile spent 6.6% of their income on insurance in 2023, compared to just 1.2% for the highest quartile.
- Integrity: Audit agents identified synthetically inserted errors with a 96.5% F1 score, proving they can serve as a scalable form of oversight for autonomous research.
Limitations and Future Work
Despite its success, the authors note several hurdles:
- Cost: Running these multi-agent loops is expensive, requiring significant LLM API and computational resources.
- Domain Scope: The current system is purely computational; it cannot perform physical experiments in a "wet lab."
- Human Bandwidth: While steering reduces micro-management, it still requires high-level expert attention, which can be a bottleneck.
Future work aims to enable parallel hypothesis exploration (running multiple branches at once) and richer multi-agent critiques to further reduce the burden on the human scientist while maintaining rigorous standards of evidence.
Summary updated