Predicting enhancers with deep convolutional neural networks

Xu Min, Wanwen Zeng, Shengquan Chen, Ning Chen, Ting Chen, Rui JiangBMC Bioinformatics 2017 · OpenReview

TL;DR

This paper introduces DeepEnhancer, a deep convolutional neural network (CNN) that predicts DNA enhancers solely from their sequence. It outperforms traditional methods and can even learn interpretable sequence motifs, paving the way for more efficient and accurate genomic analysis.

What Problem Does This Paper Address?

Identifying enhancers, crucial DNA regions that regulate gene expression, is vital for understanding biological processes and diseases. While experimental methods like ChIP-seq have mapped many enhancers, they are costly and time-consuming for large-scale, tissue-specific analyses. Existing computational methods often rely on pre-defined sequence features (like k-mers), which may not capture the full complexity of enhancer function. The paper aims to develop a more accurate and efficient computational approach that directly learns from DNA sequences.

Methods

The core of DeepEnhancer is a deep convolutional neural network (CNN). Unlike traditional methods that require manual feature engineering, DeepEnhancer uses convolutional kernels to automatically learn relevant sequence patterns (motifs) directly from the DNA sequence. The DNA sequence is treated as a 1D image with four channels, representing the four nucleotides (A, C, G, T).

Key architectural components include:

  • Convolutional Layers: Scan the input sequence to detect local patterns.
  • Batch Normalization: Stabilizes and accelerates training by normalizing layer inputs.
  • Max-Pooling Layers: Reduce the dimensionality of feature maps, making the model more robust to small shifts and reducing computational load.
  • Fully Connected Layers: Combine learned features for final classification.
  • Dropout: A regularization technique to prevent overfitting.
  • Transfer Learning: The model is first trained on a large dataset of "permissive" enhancers (those active across many cell types) and then fine-tuned on smaller datasets of cell-line-specific enhancers. This leverages knowledge gained from the larger dataset to improve performance on more specialized tasks.
  • Motif Visualization: The authors developed a method to visualize the learned convolutional kernels as sequence logos, allowing for interpretation of the motifs the CNN has learned.

Datasets

  • Training/Evaluation (Permissive Enhancers): 43,011 permissive enhancers from the FANTOM5 project.
  • Negative Samples: Randomly sampled sequences from the human reference genome (GRCh37/hg19), excluding known regulatory regions.
  • Cell-Line Specific Enhancers: Datasets for 9 cell lines (GM12878, H1-hESC, HepG2, HMEC, HSMM, HUVEC, K562, NHEK, NHLF) from the ENCODE project.

Key Insights and Ideas

  • End-to-End Learning from Sequence: DeepEnhancer demonstrates that high-accuracy enhancer prediction can be achieved by directly learning from raw DNA sequences, bypassing the need for handcrafted features.
  • Power of CNNs for Genomics: The study highlights the effectiveness of CNNs in capturing complex, hierarchical patterns within genomic sequences, similar to their success in image recognition.
  • Importance of Architectural Choices: Max-pooling and batch normalization were found to significantly improve classification performance.
  • Transfer Learning for Specificity: Transfer learning is a powerful strategy for adapting a general enhancer prediction model to specific cell types, especially when cell-line-specific data is limited.
  • Interpretable Deep Learning: Visualizing convolutional kernels as sequence logos provides a crucial link between the black-box nature of deep learning and biological interpretability, revealing biologically relevant motifs.

Key Results

  • Superior Performance: DeepEnhancer models consistently outperformed the traditional gkmSVM method in predicting both permissive and cell-line-specific enhancers.
  • AUROC/AUPRC: The best performing DeepEnhancer model (4conv2pool4norm) achieved a mean AUROC of 0.916 and AUPRC of 0.917 on the permissive enhancer dataset. For cell-line-specific enhancers, DeepEnhancer achieved an average AUROC improvement of about 7% over gkmSVM.
  • Efficiency: DeepEnhancer training was significantly faster than gkmSVM, with training epochs taking around 376 seconds for the 4conv2pool4norm model, and total training time under 2 hours, compared to gkmSVM's 6 hours.
  • Motif Discovery: The visualization of learned kernels revealed motifs similar to known transcription factor binding sites (e.g., NF-κB in GM12878 cells), validating the model's ability to learn biologically meaningful patterns.

Limitations and Future Work

  • Fixed-Length Sequences: Standard CNNs are not ideal for handling variable-length sequences. The paper addresses this with data augmentation but suggests that incorporating Recurrent Neural Networks (RNNs) like LSTMs could better handle variable lengths and long-term dependencies.
  • Nucleotide-Level Input: The current model only uses nucleotide information. Future work could involve incorporating other genomic information, such as k-mer embeddings or multiple sequence alignments, to enrich the input representation.
  • Deeper Architectures: While deeper models were explored, they didn't always yield better results, suggesting that more training data might be needed for very deep architectures.
  • New Motif Discovery: The authors acknowledge that not all learned motifs perfectly match existing databases, suggesting that their framework might uncover novel, experimentally unverified motifs.

Summary updated

Abstract

Background With the rapid development of deep sequencing techniques in the recent years, enhancers have been systematically identified in such projects as FANTOM and ENCODE, forming genome-wide landscapes in a series of human cell lines. Nevertheless, experimental approaches are still costly and time consuming for large scale identification of enhancers across a variety of tissues under different disease status, making computational identification of enhancers indispensable. Results To facilitate the identification of enhancers, we propose a computational framework, named DeepEnhancer, to distinguish enhancers from background genomic sequences. Our method purely relies on DNA sequences to predict enhancers in an end-to-end manner by using a deep convolutional neural network (CNN). We train our deep learning model on permissive enhancers and then adopt a transfer learning strategy to fine-tune the model on enhancers specific to a cell line. Results demonstrate the effectiveness and efficiency of our method in the classification of enhancers against random sequences, exhibiting advantages of deep learning over traditional sequence-based classifiers. We then construct a variety of neural networks with different architectures and show the usefulness of such techniques as max-pooling and batch normalization in our method. To gain the interpretability of our approach, we further visualize convolutional kernels as sequence logos and successfully identify similar motifs in the JASPAR database. Conclusions DeepEnhancer enables the identification of novel enhancers using only DNA sequences via a highly accurate deep learning model. The proposed computational framework can also be applied to similar problems, thereby prompting the use of machine learning methods in life sciences.
BibTeX
@article{min2017predicting,
  author = {Xu Min and Wanwen Zeng and Shengquan Chen and Ning Chen and Ting Chen and Rui Jiang},
  title = {{Predicting enhancers with deep convolutional neural networks}},
  journal = {BMC Bioinformatics},
  year = {2017},
  url = {https://openreview.net/forum?id=CqqUulzB6u7},
}
Figure