Abstract
Single-cell RNA sequencing has transformed our ability to identify diverse cell types and their transcriptomic signatures. However, annotating these signatures—especially those involving poorly characterized genes—remains a major challenge. Traditional methods, such as Gene Set Enrichment Analysis (GSEA), depend on well-curated annotations and often perform poorly in these contexts. Large Language Models (LLMs) offer a promising alternative but struggle to represent complex biological knowledge within structured ontologies. To address this, we present BRAINCELL-AID, a novel multi-agent AI system that integrates free-text descriptions with ontology labels to enable more accurate and robust gene set annotation. By incorporating retrieval-augmented generation (RAG), we developed a robust agentic workflow that refines predictions using relevant PubMed literature, reducing hallucinations and enhancing interpretability. Using this workflow, we achieved correct annotations for 77% of mouse gene sets among their top predictions. Applying this approach, we annotated 5,322 brain cell clusters from the comprehensive mouse brain cell atlas generated by the BRAIN Initiative Cell Census Network, enabling novel insights into brain cell function by identifying region-specific gene co-expression patterns and inferring functional roles of gene ensembles. BRAINCELL-AID also identifies Basal Ganglia-related cell types with neurologically meaningful descriptions. Hence, we create a valuable resource to support community-driven cell type annotation.
Introduction
Modern neuroscience is characterized by an explosion of data. Techniques like single-cell RNA sequencing (scRNA-seq) allow researchers to identify thousands of distinct cell clusters in the brain based on their genetic signatures. However, identifying a cluster is only the first step; understanding its functional role remains a significant bottleneck.
Traditionally, researchers have relied on Gene Set Enrichment Analysis (GSEA) to label these clusters. However, GSEA is limited by its reliance on static, often outdated databases and frequently yields generic labels that lack specific neurological context. While Large Language Models (LLMs) like GPT-4 show promise in biological reasoning, they are prone to "hallucinations"—generating plausible but incorrect biological facts. BRAINCELL-AID addresses these challenges by creating a system that is both biologically intelligent and strictly grounded in peer-reviewed literature.
Methods: The Multi-Agent Architecture
The core of BRAINCELL-AID is a multi-agent architecture designed to emulate the workflow of a human researcher. The system integrates three specialized agents within an agentic network:
- The Query Agent: Utilizes a strategy called GPTON (Generative Pretrained Transformer enhanced with Ontology Narration). Instead of training the LLM on rigid, structured data, Gene Ontology (GO) terms are "verbalized" into natural language descriptions. This aligns with the pre-training of LLMs, allowing the model (based on Llama 3) to better interpret gene co-functionality.
- The Literature Agent: Acts as a specialized librarian, performing semantic searches across PubMed to retrieve the most relevant abstracts related to the gene sets and initial annotations.
- The RAG Agent: Employs Retrieval-Augmented Generation to synthesize the initial predictions from the Query Agent with the evidence retrieved by the Literature Agent. This agent, powered by the Llama 3 405B model, produces a final, grounded annotation.
Key Insights and Results
The system was evaluated using over 7,000 well-annotated gene sets from the Molecular Signatures Database (MSigDB) and the Reactome dataset. It was then applied to the Allen Brain Cell (ABC) Atlas, which contains 5,322 distinct cell clusters from the mouse brain.
Performance and Accuracy
BRAINCELL-AID demonstrated a significant improvement over existing tools. In testing, 77% of mouse gene sets and 74% of human gene sets received highly relevant biological annotations within the top 5 predictions. The fine-tuned Llama 3 70B model outperformed GPT-4 and other specialized models like GeneAgent, showing a 14-20% improvement in ROUGE scores.
Biological Discovery
- Ontology Narration: Converting structured biological data into narrative form before fine-tuning significantly enhances LLM performance by leveraging their linguistic strengths.
- Functional States vs. Cell Types: The system helped clarify that some "distinct" clusters are actually the same cell type in different functional states, such as homeostatic versus activated inflammatory states.
- Contextual Inference: Unlike keyword-based tools, BRAINCELL-AID inferred labels like "Basal Ganglia" by understanding the spatial context and literature associated with the genes, even when those specific terms were absent from the underlying ontology.
Discussion and Future Directions
While BRAINCELL-AID represents a major advancement, it remains dependent on the quality of available PubMed literature. If a gene is poorly studied, the system's performance may be limited. Furthermore, the authors emphasize a "human-in-the-loop" approach, where the AI serves as a collaborative tool for experts. To facilitate this, a community web portal has been established for researchers to refine and validate the AI-generated annotations.
Future work will focus on expanding the training corpus to include broader biological knowledge and applying this framework to human disease research and drug discovery.
Summary updated