Abstract
While general-purpose Large Language Models (LLMs) have demonstrated remarkable capabilities in natural language processing, their application to specialized biological domains is often hindered by hallucinations and an inability to process non-textual modalities. This paper introduces Geneverse, a collection of open-source Multimodal Large Language Models (MLLMs) specifically finetuned for genomics and proteomics. By leveraging supervised finetuning, multimodal architectures, and tool-augmented post-processing, Geneverse models bridge the gap between the Central Dogma of biology and artificial intelligence. Our results demonstrate that these specialized models, despite their smaller scale, frequently outperform state-of-the-art closed-source models like GPT-4 in tasks such as gene function description, protein classification, and marker gene identification.
Introduction
The "Central Dogma" of molecular biology—the flow of genetic information from DNA to RNA to protein—is the cornerstone of life sciences. However, the data generated at each stage of this process is highly specialized and often multimodal, ranging from genomic sequences to complex 3D protein structures. General-purpose LLMs struggle with this complexity, often hallucinating specific gene identifiers or failing to interpret spatial transcriptomic maps.
Geneverse addresses these limitations by adapting open-source base models (including LLaMA2, LLaMA3, Mistral, and Gemma) into specialized biological assistants. By treating biological images as a distinct language and grounding model outputs in verified databases, Geneverse provides a robust framework for genomic and proteomic research.
Methodology
The development of Geneverse utilized several advanced adaptation techniques to ensure domain-specific accuracy and multimodal capability:
Supervised Instruct Finetuning (SIFT)
To specialize the models without the prohibitive cost of full parameter retraining, the researchers employed Parameter-Efficient FineTuning (PEFT), specifically LoRA (Low-Rank Adaptation). This allowed for the injection of biological knowledge into models like Mistral-7B and LLaMA3-8B while maintaining their underlying linguistic capabilities.
Multimodality via LLaVA
For tasks requiring the interpretation of protein structures and spatial data, the team utilized the LLaVA architecture. This approach allows the model to process visual data—such as AlphaFold2 protein structures—alongside text, effectively translating visual biological features into functional descriptions.
Tool-Augmentation and Post-processing
A significant challenge in biological AI is the precise handling of numerical identifiers (e.g., Ensembl or HGNC IDs). Geneverse implements a post-processing system that detects these IDs in the model's output and cross-references them with public databases to ensure factual integrity.
Dual-Grader Evaluation
Models were evaluated using a dual-grader system that assessed both truthfulness (factual accuracy) and structural correctness (adherence to formatting requirements). This ensured that the models were not only accurate but also usable in automated research pipelines.
Data and Training
The training of Geneverse relied on a strategic mix of high-quality biological data and synthetic augmentation:
- NCBI Data: Real-world gene information was sourced from the National Center for Biotechnology Information.
- Synthetic Augmentation: GPT-3.5 was used to rephrase technical database entries into natural language, improving the diversity and readability of the training set.
- AlphaFold2 & Spatial Transcriptomics: Protein structure images and human breast tissue spatial data provided the foundation for the models' multimodal training.
Key Insights
- The Hybrid Advantage: The combination of raw database text and GPT-3.5-rephrased synthetic data proved superior to using either in isolation. The synthetic data improved the models' conversational flow, while the real data provided the necessary factual grounding.
- Specialization vs. Scale: Well-tuned 7B and 13B parameter models can outperform massive models like GPT-4 in niche scientific domains. This suggests that for specialized research, domain-specific finetuning is more critical than raw parameter count.
- The Role of RAG: Interestingly, Retrieval-Augmented Generation (RAG) was found to occasionally degrade performance if the retrieved context was poorly structured, highlighting the need for precise context management in biological tasks.
Results
Geneverse models demonstrated significant performance gains across three key benchmarks:
- Gene Function Description: Mistral-7B and LLaMAPro-8B (with LoRA) achieved an average score of 0.975, surpassing GPT-4’s score of 0.925.
- Protein Classification: While GPT-4v and Gemini struggled with factual accuracy in protein structure interpretation (scoring near 0), the finetuned Geneverse LLaVA-7B achieved a score of 0.645.
- Marker Gene Identification: Geneverse LLaVA-7B reached a score of 0.95, significantly outperforming GPT-4v’s score of 0.35.
Conclusion and Future Work
Geneverse demonstrates the power of open-source, multimodal models in advancing biomedical research. By specializing models for the specific data types found in genomics and proteomics, the researchers have created a toolset that rivals and often exceeds the performance of the world's most advanced general-purpose AIs. Future iterations will focus on expanding the task variety and exploring the scaling laws of multimodal biological models.
Summary updated