Introduction
Modern biomedical research is increasingly defined by a fragmented landscape of massive datasets, specialized software, and disparate databases. While large language model (LLM) agents have shown promise in automating research tasks, most existing systems are "static." They rely on manually curated, fixed sets of tools that quickly become outdated as new bioinformatics resources are released. This creates a significant bottleneck, requiring human intervention to integrate new capabilities into an agent's repertoire.
To address this, we introduce STELLA (Self-Evolving LLM Agent), a multi-agent system designed to autonomously identify capability gaps and build its own tools. By moving from "AI as a tool" to "AI as a tool-maker," STELLA can keep pace with the cutting edge of biomedical science.
The STELLA Architecture
STELLA utilizes a multi-agent architecture composed of four specialized roles that work in an iterative loop to solve complex problems:
- Manager Agent: Acts as the central coordinator, establishing a multi-step "Reasoning Pathway" for every task.
- Developer (Dev) Agent: The execution arm that manages Conda environments and writes Python code for data analysis.
- Critic Agent: A reviewer that assesses intermediate results, identifying flaws in logic or gaps in the system's current capabilities.
- Tool Creation Agent: The innovator that searches external resources like GitHub and PubMed to discover, test, and integrate new tools into the system's "Tool Ocean."
This architecture is driven by two core mechanisms: a Template Library that distills successful workflows for future use, and a Tool Ocean—a dynamic, expanding collection of executable capabilities. When the Critic identifies a missing function (e.g., a specific virtual screening tool), the Tool Creation Agent builds or integrates it on the fly, ensuring the system is never limited by a predefined function set.
Evaluation Benchmarks
We evaluated STELLA against several high-difficulty benchmarks designed to test the limits of AI in the life sciences:
- Humanity’s Last Exam (HLE): Biomedicine: A subset of 50 questions spanning 14 subdisciplines, including Genomics and Computational Biology.
- LAB-Bench (DBQA & LitQA): A benchmark focused on Database Question-Answering and Literature Question-Answering, testing the agent's ability to navigate complex scientific documentation.
Experimental Results
STELLA consistently established a new state-of-the-art (SOTA) across all tested benchmarks, outperforming general-purpose models like Gemini 2.5 Pro and Claude 4 Opus, as well as domain-specific agents like Biomni.
Key performance metrics include:
- HLE: Biomedicine: Achieved 26% accuracy, significantly leading the field.
- LAB-Bench: Scored 54% on DBQA and 63% on LitQA.
One of the most significant findings is the correlation between "computational budget" (the number of iterative trials) and accuracy. Unlike standard LLMs, STELLA’s performance improves as it is given more "thinking time" to refine its tools and strategies. On the HLE benchmark, accuracy nearly doubled (from 14% to 26%) as the number of trials increased from 1x to 9x. This demonstrates that the self-evolving architecture allows the agent to systematically improve its performance through experience.
Discussion and Future Directions
STELLA represents a shift toward autonomous "AI Scientists" capable of bridging the gap between computational prediction and research needs. By learning from failure—identifying when an analysis is descriptive but not actionable and building the necessary tools to fix it—STELLA demonstrates a unique capacity for growth.
However, challenges remain in transitioning from digital reasoning to real-world laboratory applications. Future work will focus on deploying STELLA within actual research workflows to handle the noise of real-world data and enhancing the collaboration interface between human scientists and the agent. The ultimate goal is a system that can move seamlessly from computational hypothesis generation to wet-lab validation.
Summary updated