GENA-Web - GENomic Annotations Web Inference using DNA language models

Alexey Shmelev, Maxim Petrov, Dmitry Penzar, Nikolay Akhmetyanov, Maksim Tavritskiy, Stepan Mamontov, Yuri Kuratov, Mikhail Burtsev, Olga Kardymon, Veniamin FishmanbioRxiv 2024 · DOI

TL;DR

GENA-Web is a new web platform that makes powerful DNA language models accessible for genomic annotation tasks like predicting promoters, splice sites, and enhancer activity. It addresses the challenge of applying these advanced ML models by providing an easy-to-use interface and handling long DNA sequences, democratizing genomic research.

What Problem Does This Paper Address?

As sequencing costs plummet, we're generating vast amounts of genomic data. However, annotating these genomes and understanding the functional implications of genetic variations remains a significant hurdle. While machine learning, particularly transformer-based DNA language models, has shown immense promise in tackling these challenges, applying them in practice is often difficult. Researchers may not share their fine-tuned models, and using them typically requires advanced programming skills. This creates a gap between the potential of these powerful tools and their practical adoption in the broader research community. GENA-Web aims to bridge this gap by offering a user-friendly web service for genomic annotation using these state-of-the-art models.

Methods

GENA-Web leverages pre-trained DNA language models, specifically DNABERT and the GENA-LM family. These models are based on transformer architectures and are pre-trained on large DNA sequences to learn fundamental patterns. The core innovation here is not the models themselves, but their integration into a web service.

Key methodological aspects include:

  • Web Service Implementation: A robust front-end (TypeScript, React) and back-end (Flask, Python) architecture, with each model encapsulated in a Docker container for scalability and isolation. This allows for standardized inputs and outputs in common genomic formats (.bed, .bedGraph).
  • Handling Long Sequences: A significant challenge in genomics is the length of DNA sequences. GENA-Web employs strategies to handle sequences up to 1 megabase (Mb). For sequences exceeding model capacity, it uses overlapping chunks to maintain contextual information, ensuring accurate predictions even for very long inputs.
  • Fine-tuning: The pre-trained models are fine-tuned for specific downstream tasks like promoter activity prediction, splice site annotation, epigenetic profiling, and enhancer activity scoring.
  • Interpretability: GENA-Web incorporates the Layer Integrated Gradients method to provide insights into why a prediction was made. It highlights specific DNA tokens that contribute most to the annotation, offering a degree of interpretability often lacking in black-box ML models.

Datasets

The models used in GENA-Web were pre-trained on the human hg38 and T2T genome assemblies. For fine-tuning specific tasks, the paper mentions using:

  • EPDnew-derived promoter sequences (for promoter activity prediction).
  • Original datasets provided by authors for other tasks, referencing specific studies (e.g., for epigenetic features and enhancer activity).
  • For cross-species splice site annotation, random gene sequences were sourced from human, mouse, zebrafish, and fruit fly genomes via the UCSC genome browser.

Key Insights and Ideas

  • Democratizing Advanced ML: The most significant contribution is making sophisticated DNA language models accessible to a wider audience through a user-friendly web interface, lowering the barrier to entry for genomic research.
  • Long Sequence Handling is Crucial: The ability to process very long DNA sequences (up to 1 Mb) is a key differentiator, enabling more comprehensive genomic analyses than previous tools that were limited to shorter sequences.
  • Interpretability Matters: Providing token attribution scores helps researchers understand the sequence determinants driving predictions, moving beyond simple black-box outputs and fostering trust and deeper biological insights.
  • Cross-Species Transfer Learning: The study demonstrates that models fine-tuned on human data can still achieve reasonable accuracy in predicting splice sites in other species, suggesting conserved sequence grammar for these elements.

Key Results

  • High Accuracy in Human Tasks: GENA-Web offers tools for promoter activity prediction, splice site annotation, epigenetic profiling, and enhancer activity scoring, with models trained on human data.
  • Cross-Species Splice Site Prediction: The model trained on human data achieved high accuracy for human (93% donor, 88% acceptor), mouse (81% donor, 86% acceptor), and zebrafish (86% donor, 76% acceptor) splice sites.
  • Drosophila Performance: Accuracy for fruit fly splice site prediction was lower (54% donor, 61% acceptor), indicating species-specific differences or limitations in transferability.
  • Improved Performance over Random: In all cross-species tests, GENA-Web's predictions were significantly more accurate than random site selection.
  • Token Attribution Insights: The tool can highlight specific 6-8 bp sequence motifs that are critical for predicted features, offering detailed biological insights.

Limitations and Future Work

  • Species Bias: While some cross-species inference is possible, many of the models are trained on human data, potentially limiting their applicability to other species, especially for complex features.
  • Limited Task Scope (Currently): GENA-Web currently offers four main tasks. The authors emphasize the need to integrate more models and tasks to expand its functionality.
  • Focus on Human Chromatin: A significant portion of the current outputs pertains to chromatin states, primarily based on human data.
  • Future Directions: The authors suggest expanding GENA-Web's capabilities by integrating new models and tasks, thereby enhancing its applicability across a wider range of genomic research areas. They also highlight the potential for further development in understanding sequence determinants.

Summary updated

Abstract

ABSTRACT The advent of advanced sequencing technologies has significantly reduced the cost and increased the feasibility of assembling high-quality genomes. Yet, the annotation of genomic elements remains a complex challenge. Even for species with comprehensively annotated reference genomes, the functional assessment of individual genetic variants is not straightforward. In response to these challenges, recent breakthroughs in machine learning have led to the development of DNA language models. These transformer-based architectures are designed to tackle a wide array of genomic tasks with enhanced efficiency and accuracy. In this context, we introduce GENA-Web, a web-based platform that consolidates a suite of genome annotation tools powered by DNA language models. The version of GENA-Web presented here encompasses a diverse set of models trained on human data, including the prediction of promoter activity, annotation of splice sites, determination of various chromatin features, and a model for scoring of enhancer activity in Drosophila. GENA-Web is accessible online at https://dnalm.airi.net/
BibTeX
@misc{shmelev2024gena,
  author = {Alexey Shmelev and Maxim Petrov and Dmitry Penzar and Nikolay Akhmetyanov and Maksim Tavritskiy and Stepan Mamontov and Yuri Kuratov and Mikhail Burtsev and Olga Kardymon and Veniamin Fishman},
  title = {{GENA-Web - GENomic Annotations Web Inference using DNA language models}},
  year = {2024},
  doi = {10.1101/2024.04.26.591391},
  howpublished = {bioRxiv},
}
Figure