Abstract
Transcription factors are proteins that regulate the expression of genes by binding to specific genomic regions known as Transcription Factor Binding Sites (TFBSs), typically located in the promoter regions of those genes. Accurate prediction of these binding sites is essential for understanding the complex gene regulatory networks underlying various cellular functions. In this work, we have developed a deep learning model which uses pre-trained DNABERT, a Convolutional Neural Network (CNN) module, a Modified Convolutional Block Attention Module (MCBAM), a Multi-Scale Convolutions with Attention (MSCA) module and an output module. The pre-trained DNABERT is used for sequence embedding, thereby capturing the long-term dependencies in the DNA sequences while the CNN, MCBAM and MSCA modules are useful in extracting higher-order local features. TFBS-Finder is trained and tested on 165 ENCODE ChIP-seq datasets. We have also performed ablation studies as well as cross-cell line validations and comparisons with other models. The experimental results show the superiority of the proposed method in predicting TFBSs compared to the existing methodologies.
Introduction
Transcription factors (TFs) act as critical regulators of the genetic code, binding to specific DNA sequences called Transcription Factor Binding Sites (TFBSs) to control gene expression. Mapping these sites is vital for understanding biological processes and disease mechanisms. While experimental methods like ChIP-seq are highly accurate, they are costly and time-consuming. Computational models offer a scalable alternative, yet they must overcome the challenge of capturing both short, local motifs and long-range genomic dependencies. Existing models often struggle to balance these two scales effectively.
Methodology
The TFBS-Finder architecture is a five-module pipeline designed to integrate global context-awareness with specialized local feature extraction.
The model's components include:
- DNABERT: A pre-trained transformer that treats DNA as a language, providing embeddings that capture global context and long-range dependencies.
- CNN Module: A convolutional layer that extracts high-order local features from the DNABERT embeddings.
- Modified Convolutional Block Attention Module (MCBAM): This module applies spatial and channel attention to highlight important sequence regions and features. The authors found that applying spatial attention before channel attention optimized performance.
- Multi-Scale Convolutions with Attention (MSCA): Uses varying kernel sizes to detect patterns of different lengths simultaneously.
- Output Module: Fuses features from the MCBAM and MSCA modules to generate the final prediction.
Experimental Setup and Datasets
The model was rigorously evaluated using 165 ENCODE ChIP-seq datasets, covering 29 transcription factors across 32 human cell lines. Positive samples were 101-bp DNA sequences, while negative samples were generated by shuffling positive sequences while preserving dinucleotide frequencies to maintain biological realism.
Results and Discussion
TFBS-Finder consistently outperformed existing state-of-the-art models. It achieved an average accuracy of 0.930, a PR-AUC of 0.961, and a ROC-AUC of 0.961. Notably, it surpassed the previous leading model, BERT-TFBS, by 7.9% in accuracy.
Ablation studies confirmed that the combination of Transformers and CNN-based attention modules is highly effective; DNABERT handles global logic, while the attention modules act as refined local motif detectors. Furthermore, the model demonstrated robustness in cross-cell line validations, maintaining high performance (ROC-AUC > 0.93) across different cell types, suggesting it learns general biological rules rather than dataset-specific artifacts.
Conclusion
TFBS-Finder represents a significant advancement in the computational prediction of transcription factor binding sites. By leveraging the strengths of both transformer-based global embeddings and multi-scale attention-driven local feature extraction, it achieves superior accuracy and robustness. Future research will investigate the incorporation of 3D DNA shape information to further improve predictive performance.
Summary updated