Language may be all omics needs: Harmonizing multimodal data for omics understanding with CellHermes

Yicheng Gao, Weixu Wang, Yuheng Zhao, Kejing Dong, Caihua Shan, Weizhong Zheng, Till Richter, Zekai Li, Siming Chen, Fabian J. Theis, Qi LiubioRxiv 2025 · DOI

TL;DR

CellHermes is a novel biological language model that unifies diverse omics data (like gene expression and protein interactions) by translating them into natural language. This allows a single, pre-trained Large Language Model (LLM) to act as an encoder for biological entities, a predictor for multi-task biological questions, and an explainer for biological mechanisms, achieving competitive or superior performance to existing single-modality models.

What Problem Does This Paper Address?

Decoding complex cellular systems requires integrating vast amounts of data from various "omics" (genomics, transcriptomics, proteomics, etc.). However, current models are often trained on a single data type (modality) and from scratch, limiting their ability to generalize and understand the interconnectedness of biological processes. This siloed approach hinders a holistic understanding of cellular function. The paper addresses the challenge of integrating these diverse, multimodal omics datasets and leveraging the power of pre-trained LLMs to create a more unified, interpretable, and generalizable framework for biological research.

Methods

CellHermes builds upon existing LLMs (specifically LLaMA-3.1-8B-Instruct) by using parameter-efficient LoRA (Low-Rank Adaptation) fine-tuning. The core innovation lies in reformulating diverse omics data into a question-answer (instruction) format. This allows the LLM to emulate various self-supervised learning (SSL) techniques:

  • For transcriptomic data: Masked Language Modeling (MLM) and Autoregressive Prediction are used, treating gene expression profiles as "sentences" of ranked gene names.
  • For graph data (like PPI networks): Masked Node Prediction and Masked Link Prediction are employed, transforming network structures into natural language descriptions of relationships.

This unified, question-answer framework enables CellHermes to function in three key roles:

  1. Encoder: Generates high-dimensional embeddings for genes and cells, capturing biological context.
  2. Predictor: Unifies heterogeneous downstream tasks by translating them into natural language questions, allowing for multi-task prediction.
  3. Explainer: Leverages attention mechanisms and text-based reasoning to provide interpretable explanations for its predictions.

Datasets

The paper utilizes several datasets for training and evaluation:

  • Pretraining Corpus:
    • ~1 million single-cell transcriptomics data from CellxGene.
  • A protein-protein interaction (PPI) network involving ~20,000 genes from BioGRID.
  • Downstream Tasks & Benchmarks:
    • Gene Embeddings: Five downstream tasks inspired by previous studies (e.g., predicting transcription factor range, dosage sensitivity).
  • Cell Embeddings: Five datasets (Immune, Aorta, Lung, Pancreas, PBMC) for evaluating biological signal preservation and batch effect removal.
  • BioUniBench: A newly proposed benchmark integrating 7 databases (Perturbase, DepMap, BioGRID, CellMarker, CTD, Orphanet, DisGenNet) across 10 tasks, including perturbation response, cell fitness, gene-disease association, and cell type identification.

Key Insights and Ideas

  • Natural Language as a Unifying Medium: The paper strongly advocates for natural language as a powerful and flexible way to integrate diverse biological data and tasks, overcoming the limitations of modality-specific models.
  • LLMs for Biological Understanding: Demonstrates that pre-trained LLMs, when adapted with omics-specific data and tasks, can achieve remarkable performance in biological representation, prediction, and explanation.
  • Efficiency of LoRA Fine-tuning: Highlights that parameter-efficient LoRA fine-tuning on existing LLMs can yield competitive results with significantly less computational cost compared to training from scratch or full fine-tuning.
  • Multitask Learning Synergy: Shows that jointly training on multiple biological tasks can lead to improved performance and better generalization due to cross-task knowledge transfer.
  • Interpretability through Dual Approach: The combination of attention-based analysis and text-based reasoning provides a powerful way to understand why CellHermes makes certain predictions, bridging the gap between model performance and biological insight.

Key Results

  • Encoder Performance: CellHermes gene embeddings consistently outperformed 9 existing single-cell foundation models on gene-level downstream tasks. CellHermes-s (sentence-based cell embedding) achieved top-four performance across multiple datasets for cell embedding, demonstrating efficiency and competitive quality.
  • Predictor Performance: CellHermes achieved comparable or superior performance on BioUniBench tasks, unifying 10 tasks across 7 databases. For perturbation prediction, it outperformed baseline models in zero-shot settings and showed strong performance with instruction fine-tuning. For cell fitness prediction, it achieved superior performance with instruction fine-tuning.
  • Explainer Utility: In a melanoma patient dataset, CellHermes successfully predicted tumor reactivity and, through its explanation module, identified key genes and pathways involved in tumor-reactive T cell function.

Limitations and Future Work

  • Training Corpus Scale: The authors acknowledge that the current training corpus is modest compared to the vast amount of available biomedical data. They suggest incorporating more diverse multimodal data (proteomics, epigenomics, metabolomics, etc.) to further enhance generalization.
  • LLM Choice: CellHermes is currently built on LLaMA-3.1-8B-Instruct. Future work could involve exploring larger open-source LLMs and systematically benchmarking them.
  • Data Representation: The current method converts omics data into text, which might discard some numerical precision. Exploring other alignment techniques or reinforcement learning could be beneficial.
  • Extensibility: The framework is designed to be extensible to new biological resources and tasks, with BioUniBench being a continuously curated resource for future development and benchmarking.
  • Virtual Cells: The paper suggests that CellHermes could contribute to building virtual cells by integrating omics data and reasoning capabilities.

Summary updated

Abstract

Abstract Decoding cellular systems requires integrating diverse omics data, yet most models are trained from scratch on a single modality, restricting generalization. Here we present CellHermes, a biological language model that leverages pretrained large language models (LLMs) to integrate multimodal forms of omics data, such as transcriptomic profile and PPI network, through natural language for better omics understanding. By reformulating these datasets into question-answer pairs, CellHermes emulates multiple self-supervised learning paradigms within a single, universal, language-based framework with LORA fine-tuning on existing natural language-LLM, achieving comparable or even better performance compared to current single-cell foundation models. Within this framework, CellHermes functions as an encoder, a predictor and an explainer for supporting a range of downstream tasks. For encoder, CellHermes encodes biological entities with embeddings that enable accurate network discovery and cross-dataset generalization; For predictor, CellHermes unifies heterogeneous downstream tasks by translating them into natural language question-answers pairs, allowing a single model to perform multi-task prediction by instruction fine-tuning; For explainer, CellHermes explains molecular mechanisms by combining attention analysis with text-based reasoning, leveraging the interpretability and reasoning capabilities of LLMs. We evaluated the ability of CellHermes as an encoder on representing genes and cells using 5 gene-level downstream tasks and 5 diverse single-cell datasets across different tissues compared with other single cell foundation models. We also evaluated CellHermes as a predictor on BioUniBench, a new benchmark of 10 tasks across 7 databases for LLMs, including perturbation response, cell fitness estimation, gene-disease association, and cell type identification. All benchmarks suggest CellHermes can unify multiple biological tasks within one model without compromising performance. By applying CellHermes as an explainer for a melanoma patients dataset, it can uncover potential key genes on tumor-reactive T cells. Together, CellHermes establishes natural language as a unifying medium for omics, offering a foundation that, in the future, potentially enables a more integrated and interpretable research loop of biological representation, prediction, and interpretability.
BibTeX
@misc{gao2025language,
  author = {Yicheng Gao and Weixu Wang and Yuheng Zhao and Kejing Dong and Caihua Shan and Weizhong Zheng and Till Richter and Zekai Li and Siming Chen and Fabian J. Theis and Qi Liu},
  title = {{Language may be all omics needs: Harmonizing multimodal data for omics understanding with CellHermes}},
  year = {2025},
  doi = {10.1101/2025.11.07.687322},
  howpublished = {bioRxiv},
}
Figure