EvoScientist: Towards Multi-Agent Evolving AI Scientists for End-to-End Scientific Discovery

Yougang Lyu, Xi Zhang, Xinhao Yi, Yuyue Zhao, Shuyu Guo, Wenxiang Hu, Jan A. Piotrowski, Jakub Kaliski, Jacopo Urbani, Zaiqiao Meng, Lun Zhou, Xiaohui Yan

TL;DR

EvoScientist is a multi-agent framework that moves beyond static AI research pipelines by introducing "self-evolution." By using a dedicated Evolution Manager to distill successes and failures into persistent memory, the system continuously improves its ideation and coding strategies, eventually producing papers that have won "Best Paper" awards at AI conferences.

What Problem Does This Paper Address?

While Large Language Models (LLMs) have shown promise in automating parts of the scientific process—like brainstorming or writing code—most current "AI Scientist" systems are static. They follow hand-designed, fixed pipelines. If a specific research direction fails or a coding strategy proves inefficient, the system doesn't "learn" from that mistake for the next task; it simply resets.

This lack of persistent memory leads to several issues: systems repeat failed experiments, pursue scientifically infeasible ideas, and fail to adapt their strategies based on accumulated experience. The authors of EvoScientist aim to transform end-to-end scientific discovery from a static execution task into a continuous learning problem.

Methods

EvoScientist employs a trio of specialized agents coordinated by a self-evolution loop:

  1. Researcher Agent (RA): Responsible for ideation. It performs a "tree search" (propose-review-refine) to generate candidate ideas, grounded in literature and retrieved "direction insights" from memory. It uses an Elo-based tournament system to rank and select the best ideas.
  2. Engineer Agent (EA): Handles the "grunt work" of implementation. It executes experiments in four stages: initial implementation, hyperparameter tuning, method execution, and ablation studies. It retrieves successful code patterns from memory to increase its success rate.
  3. Evolution Manager Agent (EMA): The "brain" of the evolution process. After a task is completed, the EMA analyzes the interaction history. It distills promising research directions into Ideation Memory and successful data/model strategies into Experimentation Memory. Crucially, it also records failed directions to ensure the system doesn't revisit them.

The system uses a "persistent memory" architecture where these two memory modules ($M_I$ and $M_E$) are updated after every research cycle, allowing the agents to retrieve relevant prior strategies for new, unseen tasks.

Datasets

To evaluate the system, the authors constructed a multi-level evaluation set:

  • Idea Generation: A curated set of 30 research queries spanning topics like Machine Translation, Healthcare Agents, and RAG, solicited from experienced AI researchers.
  • Code Generation: The research proposals generated from the 30 queries were used as inputs to test the system's ability to operationalize ideas.
  • End-to-End Discovery: 6 specific research ideas were developed into full manuscripts and submitted to the ICAIS 2025 (International Conference on AI Scientists) for peer review.

Key Insights and Ideas

  • Learning from Failure is Key: One of the most important features is "Idea Validation Evolution." By recording why a proposal failed (e.g., the method performed worse than the baseline), the system filters out infeasible directions in future runs.
  • The Power of Distillation: Instead of just saving raw logs, the Evolution Manager summarizes insights. This turns a messy execution trace into "actionable" knowledge that a future agent can easily inject into its prompt.
  • Empirical vs. Theoretical: The system is exceptionally strong at empirical discovery—finding "what" works through rigorous experimentation—but the authors note it still struggles with the "why" (deep theoretical formalization), which remains a primary domain for human researchers.

Key Results

  • Superior Ideation: EvoScientist outperformed 7 state-of-the-art baselines (including AI Scientist-v2 and commercial tools like Novix) across novelty, feasibility, relevance, and clarity.
  • Improved Reliability: Through multi-agent evolution, the mean code execution success rate jumped from 34.39% to 44.56%.
  • Real-World Validation: All 6 papers generated by the system were accepted at ICAIS 2025. Notably, one received the Best Paper Award and another received the AI Reviewer’s Appraisal Award.
  • Ablation Success: Removing the evolution components (the memory updates) led to a significant drop in novelty and feasibility, proving that the "learning" mechanism is what drives the performance gains.

Limitations and Future Work

  • Domain Constraint: The current evaluation is limited to computational research (code and simulation). Expanding to "wet labs" (materials science, biology) would require integrating the framework with robotic laboratory workflows.
  • Theoretical Depth: As noted by conference reviewers, the system tends to prioritize testable empirical results over deep theoretical proofs. Future work could involve agents specialized in mathematical formalization.
  • Ethical Guardrails: The authors acknowledge the risks of "dual-use" (e.g., using AI to discover harmful substances) and emphasize the need for human-in-the-loop verification and safety safeguards in autonomous discovery systems.

Summary updated

Abstract

The increasing adoption of Large Language Models (LLMs) has enabled AI scientists to perform complex end-to-end scientific discovery tasks requiring coordination of specialized roles, including idea generation and experimental execution. However, most state-of-the-art AI scientist systems rely on static, hand-designed pipelines and fail to adapt based on accumulated interaction histories. As a result, these systems overlook promising research directions, repeat failed experiments, and pursue infeasible ideas. To address this, we introduce EvoScientist, an evolving multi-agent AI scientist framework that continuously improves research strategies through persistent memory and self-evolution. EvoScientist comprises three specialized agents: a Researcher Agent (RA) for scientific idea generation, an Engineer Agent (EA) for experiment implementation and execution, and an Evolution Manager Agent (EMA) that distills insights from prior interactions into reusable knowledge. EvoScientist contains two persistent memory modules: (i) an ideation memory, which summarizes feasible research directions from top-ranked ideas while recording previously unsuccessful directions; and (ii) an experimentation memory, which captures effective data processing and model training strategies derived from code search trajectories and best-performing implementations. These modules enable the RA and EA to retrieve relevant prior strategies, improving idea quality and code execution success rates over time. Experiments show that EvoScientist outperforms 7 open-source and commercial state-of-the-art systems in scientific idea generation, achieving higher novelty, feasibility, relevance, and clarity via automatic and human evaluation. EvoScientist also substantially improves code execution success rates through multi-agent evolution, demonstrating persistent memory's effectiveness for end-to-end scientific discovery.
BibTeX
@misc{lyu2026evoscientist,
  author = {Yougang Lyu and Xi Zhang and Xinhao Yi and Yuyue Zhao and Shuyu Guo and Wenxiang Hu and Jan A. Piotrowski and Jakub Kaliski and Jacopo Urbani and Zaiqiao Meng and Lun Zhou and Xiaohui Yan},
  title = {{EvoScientist: Towards Multi-Agent Evolving AI Scientists for End-to-End Scientific Discovery}},
  year = {2026},
  eprint = {2603.08127},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2603.08127},
}

Loading related content...

Figure