Genomic Language Model for Predicting Enhancers and Their Allele-Specific Activity in the Human Genome

Rekha Sathian, Pratik Dutta, Ferhat Ay, Ramana V. DavuluribioRxiv 2025 · DOI

TL;DR

This paper introduces DNABERT-Enhancer, a novel method that leverages a pre-trained genomic language model (DNABERT) to predict enhancer regions in the human genome with high accuracy. By capturing complex sequence context, DNABERT-Enhancer outperforms existing methods and can identify potential functional genetic variants within these regulatory elements, offering valuable resources for genomic interpretation.

What Problem Does This Paper Address?

Understanding the function of enhancers, crucial DNA regions that regulate gene expression, is a significant challenge in genomics. Enhancers are often located far from the genes they regulate, have variable lengths, and their activity can be cell-type specific. Traditional methods struggle to accurately identify these regions due to their complex sequence features and lack of consistent epigenetic signatures. While machine learning has shown promise, many existing models focus on nucleotide composition and fail to capture the crucial sequence context, leading to suboptimal performance. This paper aims to bridge this gap by developing a more accurate enhancer prediction method that can also identify genetic variants impacting enhancer function.

Methods

The core of this work is the development of DNABERT-Enhancer, a method that fine-tunes the pre-trained DNABERT language model for enhancer prediction. DNABERT itself is based on the Bidirectional Encoder Representations from Transformers (BERT) architecture, originally developed for natural language processing, and adapted to understand the "language" of DNA sequences.

Key methodological aspects include:

  • Leveraging Pre-trained DNABERT: Instead of training a model from scratch, the authors utilize DNABERT, which has already learned rich contextual and semantic information from a large corpus of human genomic sequences. This allows the model to better understand the nuances of DNA sequences.
  • Fine-tuning for Enhancer Prediction: The pre-trained DNABERT model is then fine-tuned on a curated dataset of known enhancers and non-enhancers. This process adapts the general genomic language understanding to the specific task of identifying enhancer sequences.
  • Two Model Variants: Two DNABERT-Enhancer models were trained: one for 201bp sequences and another for 350bp sequences, reflecting common lengths of regulatory elements.
  • Genome-Wide Prediction: The fine-tuned models were applied to the entire human genome to identify novel enhancer regions.
  • Variant Effect Prediction: DNABERT-Enhancer was further used to predict the impact of genetic variants (SNPs) on enhancer activity and transcription factor binding sites. This involved re-computing predictions with altered alleles.

Datasets

The study utilized several key datasets:

  • ENCODE Registry of Candidate cis-Regulatory Elements (cCREs): This was the primary source for positive instances (enhancers). Data was curated from the ENCODE Encyclopedia.
  • Open-source enhancer databases: Additional enhancer data was collected from various public databases to supplement training and for validation.
  • Human Reference Genome (GRCh38): Used to generate negative instances (non-enhancers) by selecting regions not annotated as enhancers.
  • dbSNP database: Used to identify genetic variants (SNPs) within predicted enhancer regions for variant effect prediction.
  • ClinVar and GWAS databases: Queried to find predicted variants with known clinical significance.
  • GTEx catalog: Used to identify genetic regulatory variants affecting gene expression.

The paper also describes the creation of specific training and testing datasets (ESd-201-Train, ESd-201-Test, ESd-350-Train, ESd-350-Test) with balanced ratios of enhancers to non-enhancers.

Key Insights and Ideas

  • Genomic Language Models are Powerful for Regulatory Element Prediction: The success of DNABERT-Enhancer highlights that treating DNA sequences as a "language" and using pre-trained language models can capture complex contextual information crucial for identifying regulatory elements like enhancers.
  • Sequence Context is King: Unlike methods relying solely on nucleotide composition, DNABERT-Enhancer's ability to learn sequence context is a key driver of its improved performance. This suggests that the arrangement and relationships between nucleotides are more informative than just their presence.
  • Longer Enhancers Provide More Information: Models trained on longer sequences (350bp) generally outperformed those trained on shorter sequences (201bp), indicating that longer regions offer richer regulatory information and context.
  • Predicting Functional Variants is Feasible: The approach successfully extends beyond simple enhancer identification to predict the functional impact of genetic variants, opening doors for understanding disease mechanisms.
  • DNABERT-Enhancer Captures Novel Enhancers: The model identified a significant number of enhancer regions not present in the training ENCODE data, demonstrating its ability to generalize and discover new regulatory elements.

Key Results

  • High Accuracy: The best fine-tuned DNABERT-Enhancer model achieved 88.05% accuracy and a Matthews correlation coefficient (MCC) of 76.22% on an independent test set (ESd-350-Test).
  • Outperforms Baselines: DNABERT-Enhancer significantly outperformed traditional machine learning methods (Random Forest, KNN, SVC, etc.) and recent state-of-the-art enhancer prediction tools (iEnhancer-ELM, iEnhancer-DCLA, iEnhancer-ECNN) across various metrics.
  • Genome-Wide Coverage: The model predicted 1,822,321 enhancer regions, covering approximately 22% of the human genome.
  • Validation Against Public Databases: Approximately 28.17% of predicted enhancers mapped to existing ENCODE SCREEN regions, and a high percentage (around 99%) of predicted enhancers intersected with at least one of nine other open-source enhancer databases.
  • Variant Impact Prediction: Identified 62,592 variants with a disruptive impact on enhancers and 15,693 disruptive binding site variants for transcription factors. Further analysis linked some of these to known clinical significance.

Limitations and Future Work

  • Sequence Length Dependence: While the 350bp model performed better, the study acknowledges that enhancers can vary significantly in length, and the model's performance might be influenced by this.
  • Experimental Validation: The predicted variants are candidates requiring further experimental validation to confirm their functional impact on specific traits or diseases.
  • Future Directions: The authors anticipate that the DNABERT-Enhancer model could be applied to predict enhancers in other species (rodents, non-human primates) without extensive re-training. The approach also holds promise for prioritizing causal variants and identifying candidate enhancers for novel gene and cell therapies.

Summary updated

Abstract

ABSTRACT Predicting and deciphering the regulatory logic of enhancers is a challenging problem, due to the intricate sequence features and lack of consistent genetic or epigenetic signatures that can accurately discriminate enhancers from other genomic regions. Recent machine-learning based methods have spotlighted the importance of extracting nucleotide composition of enhancers but failed to learn the sequence context and perform suboptimally. Motivated by advances in genomic language models, we developed DNABERT-Enhancer, a novel enhancer prediction method, by applying DNABERT pre-trained language model on the human genome. We trained two different models, using large collection of enhancers curated from the ENCODE registry of candidate cis-Regulatory Elements. The best fine-tuned model achieved 88.05% accuracy with Matthews correlation coefficient of 76% on independent set aside data. Further, we present the analysis of the predicted enhancers for all chromosomes of the human genome by comparing with the enhancer regions reported in publicly available databases. Finally, we applied DNABERT-Enhancer along with other DNABERT based regulatory genomic region prediction models to predict candidate SNPs with allele-specific enhancer and transcription factor binding activity. The genome-wide enhancer annotations and candidate loss-of-function genetic variants predicted by DNABERT-Enhancer provide valuable resources for genome interpretation in functional and clinical genomics studies.
BibTeX
@misc{sathian2025genomic,
  author = {Rekha Sathian and Pratik Dutta and Ferhat Ay and Ramana V. Davuluri},
  title = {{Genomic Language Model for Predicting Enhancers and Their Allele-Specific Activity in the Human Genome}},
  year = {2025},
  doi = {10.1101/2025.03.18.644040},
  howpublished = {bioRxiv},
}
Figure