Systematic Analysis and Accurate Identification of DNA N4-Methylcytosine Sites by Deep Learning

Lezheng Yu, Yonglin Zhang, Xue Li, Fengjuan Liu, Qi Chen, Jiesi Luo, Runyu JingFrontiers in Microbiology 2022 · DOI

TL;DR

This paper systematically analyzes existing deep learning models for identifying DNA N4-methylcytosine (4mC) sites, a crucial epigenetic marker. By benchmarking different architectures, encoding methods, and attention mechanisms on standard datasets, they developed and validated "DeepDNA4mC," a hybrid CNN-RNN model with an attention mechanism that achieves state-of-the-art performance in predicting 4mC sites.

What Problem Does This Paper Address?

DNA methylation, particularly N4-methylcytosine (4mC), is a vital epigenetic modification influencing gene expression, DNA replication, and repair. Accurately identifying these modification sites across genomes is crucial for understanding their biological roles and associated diseases. While deep learning has shown promise for this task, there's a lack of systematic analysis comparing different deep learning approaches, their components (architectures, encoding), and their impact on prediction accuracy. This makes it challenging for researchers to choose the best methods or develop new, improved predictors.

Methods

The authors conducted a comprehensive benchmarking study. They first reviewed existing deep learning-based 4mC predictors. Then, they systematically evaluated the contribution of different deep learning architectures (CNN, RNN, CNN-RNN), sequence encoding methods (one-hot, dictionary), and the inclusion of an attention mechanism. They trained and tested various combinations on standard datasets and used metrics like accuracy, F-value, recall, precision, and MCC to assess performance. They also employed visualization techniques like UMAP and Deep SHAP to understand the model's internal workings and identify important sequence features.

Datasets

The study utilized two primary benchmark datasets:

  • Zeng_2020_1: A larger dataset created by merging data from A. thaliana, C. elegans, and D. melanogaster. Positive samples were from the MethSMRT database, and negative samples were non-methylated cytosines of the same length. Sequences were processed to remove redundancy.
  • Zeng_2020_2: Another dataset also used for benchmarking, comprising data from C. elegans sourced from both the Ye et al. (2016) study and the updated MethSMRT database.

These datasets were chosen for their relevance and to allow comparison with previous studies.

Key Insights and Ideas

  • Hybrid Architectures Shine: The combination of Convolutional Neural Networks (CNNs) for local feature extraction and Recurrent Neural Networks (RNNs) for sequential context (CNN-RNN) generally outperformed standalone CNNs or RNNs, suggesting that capturing both local motifs and long-range dependencies is beneficial.
  • Attention is Key: Incorporating a self-attention mechanism significantly improved the performance of CNN-RNN models, allowing them to focus on more relevant parts of the DNA sequence for prediction.
  • One-Hot Encoding Dominates: For sequence representation, the standard one-hot encoding method consistently yielded better results than dictionary encoding across various models and species.
  • Feature Importance Revealed: Visualization techniques like Deep SHAP helped identify specific sequence motifs and nucleotide positions that are most critical for predicting 4mC sites, offering insights into the underlying biological mechanisms.
  • No One-Size-Fits-All: While CNN-RNN with attention (DeepDNA4mC) performed best overall on one dataset, the CNN architecture alone showed superior performance on another dataset, highlighting the importance of considering dataset characteristics and specific model tuning.

Key Results

  • The developed DeepDNA4mC model, a CNN-RNN architecture with a one-hot encoding and attention mechanism, achieved state-of-the-art performance on the Zeng_2020_1 dataset, outperforming other existing methods in MCC scores for C. elegans (0.789) and D. melanogaster (0.749).
  • On the Zeng_2020_2 dataset, the CNN architecture, when optimally tuned, achieved the highest performance for C. elegans (ACC: 91.6%, MCC: 0.821).
  • The study demonstrated that the attention mechanism consistently improved the performance of RNN and CNN-RNN models.
  • One-hot encoding was found to be the superior encoding method compared to dictionary encoding for most tested models.
  • Deep SHAP analysis revealed that the central and downstream regions of the DNA sequence (specifically around positions -2 to +6 bp) play the most critical role in predicting 4mC sites, with distinct motif patterns identified for each species.

Limitations and Future Work

  • Dataset Specificity: While the study used standard datasets, the performance of models can still be influenced by dataset size, quality, and the specific biological context of the species. The authors note that the CNN-RNN_attention model's performance was slightly weaker than CNN on one dataset, suggesting that model choice might be context-dependent.
  • "Black Box" Nature: Despite visualization efforts, deep learning models can still be considered "black boxes." While SHAP analysis provides feature importance, fully elucidating the complex decision-making process remains an ongoing challenge.
  • Future Directions: The authors suggest that researchers should carefully consider dataset characteristics, choose appropriate deep learning architectures and encoding methods, and leverage visualization tools for better understanding and development of 4mC prediction models. They also highlight the potential for further exploration of novel motif patterns identified by SHAP analysis.

Summary updated

Abstract

DNA N<sup>4</sup>-methylcytosine (4mC) is a pivotal epigenetic modification that plays an essential role in DNA replication, repair, expression and differentiation. To gain insight into the biological functions of 4mC, it is critical to identify their modification sites in the genomics. Recently, deep learning has become increasingly popular in recent years and frequently employed for the 4mC site identification. However, a systematic analysis of how to build predictive models using deep learning techniques is still lacking. In this work, we first summarized all existing deep learning-based predictors and systematically analyzed their models, features and datasets, etc. Then, using a typical standard dataset with three species (<i>A. thaliana</i>, <i>C. elegans</i>, and <i>D. melanogaster</i>), we assessed the contribution of different model architectures, encoding methods and the attention mechanism in establishing a deep learning-based model for the 4mC site prediction. After a series of optimizations, convolutional-recurrent neural network architecture using the one-hot encoding and attention mechanism achieved the best overall prediction performance. Extensive comparison experiments were conducted based on the same dataset. This work will be helpful for researchers who would like to build the 4mC prediction models using deep learning in the future.
BibTeX
@inproceedings{yu2022systematic,
  author = {Lezheng Yu and Yonglin Zhang and Xue Li and Fengjuan Liu and Qi Chen and Jiesi Luo and Runyu Jing},
  title = {{Systematic Analysis and Accurate Identification of DNA N4-Methylcytosine Sites by Deep Learning}},
  booktitle = {Frontiers in Microbiology},
  year = {2022},
  doi = {10.3389/fmicb.2022.843425},
}
Figure