Multi-head attention-based U-Nets for predicting protein domain boundaries using 1D sequence features and 2D distance maps

Sajid Mahmud, Zhiye Guo, Farhan Quadir, Jian Liu, Jianlin ChengBMC Bioinformatics 2022 · OpenReview

TL;DR

This paper introduces DistDom, a novel deep learning framework that leverages both 1D sequence features and predicted 2D protein distance maps to accurately identify protein domain boundaries. By combining U-Nets with a multi-head attention mechanism, DistDom significantly improves upon existing methods, demonstrating its utility in both predicting boundaries and classifying proteins as single- or multi-domain.

What Problem Does This Paper Address?

Proteins are often composed of distinct structural and functional units called domains. Identifying the boundaries between these domains is crucial for understanding protein structure, function, and evolution. However, accurately predicting these boundaries solely from amino acid sequences remains a significant challenge. Existing methods often rely on template-based approaches that fail when homologous templates are scarce, or ab initio methods that struggle to extract sufficient information from sequence features alone. This paper aims to bridge this gap by developing a more robust and accurate deep learning method that incorporates structural information derived from predicted distance maps, a feature rarely utilized in this context.

Methods

DistDom employs a multi-head attention-based U-Net architecture. The core idea is to process two types of input features:

  • 1D Sequence Features: These capture evolutionary and physicochemical properties of amino acids, generated using established tools like PSI-BLAST and HHblits.
  • 2D Inter-Residue Distance Maps: These are predicted using a deep learning method (DeepDist) and provide structural information about how residues are spatially arranged. This is a novel input for domain boundary prediction.

These 1D and 2D features are fed into separate 1D and 2D U-Net architectures, respectively. The U-Nets are designed to extract hierarchical features. The outputs from these U-Nets are then concatenated and processed by a multi-head attention mechanism. This attention mechanism is key to capturing long-range dependencies within the protein sequence, which are important for identifying global domain structures. The final output is a probability for each residue being a domain boundary.

Datasets

The study utilizes several datasets:

  • Topdomain dataset: Used for training and validation (4072 protein targets).
  • CASP7-13 dataset: Used as an independent test set (619 proteins).
  • CASP14 dataset: Used as another independent test set (55 proteins), featuring high-quality, expert-annotated domain boundaries.

To ensure rigorous evaluation, proteins with high sequence identity (>30%) between the training and validation/test sets were removed.

Key Insights and Ideas

  • 2D Distance Maps as Structural Clues: The paper demonstrates that predicted 2D distance maps, which encode structural information, are highly valuable for domain boundary prediction, a feature largely underutilized by previous methods.
  • Multi-Head Attention for Long-Range Dependencies: The integration of multi-head attention is crucial for capturing long-range interactions across the protein sequence, which are essential for understanding global domain structures, outperforming traditional RNNs or LSTMs.
  • U-Nets for Hierarchical Feature Extraction: The use of U-Nets for both 1D and 2D inputs allows for effective hierarchical feature extraction, adapting well to the spatial nature of distance maps and sequential nature of protein features.
  • Impact of Normalization: Normalizing the distance maps (specifically using a 12 Å threshold and min-max normalization) significantly improves prediction performance, highlighting the importance of data preprocessing.

Key Results

  • Improved Accuracy: DistDom achieved an accuracy of 75.9% in classifying CASP14 single- and multi-domain targets, outperforming the state-of-the-art by 13.28%.
  • Higher F1 Scores: On CASP14 multi-domain targets with expert annotations, DistDom achieved an average per-target F1 score of 0.263, a 29.56% improvement over the state-of-the-art.
  • Ablation Studies: Combining 1D and 2D inputs consistently outperformed using either input alone across all tested datasets.
  • Distance Threshold Impact: A 12 Å distance threshold for processing distance maps yielded the best performance.
  • Performance on Easy vs. Hard Targets: DistDom showed comparable performance on both easy and hard targets in the CASP14 dataset, suggesting robustness.

Limitations and Future Work

  • Dataset Differences: The paper notes that performance on CASP7-13 and CASP14 datasets is lower than on the Topdomain validation set, attributing this to the more challenging and expert-curated nature of the CASP datasets.
  • Small Number of Easy Targets: The CASP14 dataset has a very small number of "easy" targets (4), making it difficult to draw definitive conclusions about performance on such cases. More data would be needed for reliable estimation.
  • Future Work: The authors suggest that the domain boundary predictions can be effectively used to classify proteins as single-domain or multi-domain, which is a direct application of their method. Further exploration of the 2D distance map's utility and the multi-head attention mechanism's capabilities could lead to even more refined models.

Summary updated

Abstract

The information about the domain architecture of proteins is useful for studying protein structure and function. However, accurate prediction of protein domain boundaries (i.e., sequence regions separating two domains) from sequence remains a significant challenge. In this work, we develop a deep learning method based on multi-head U-Nets (called DistDom) to predict protein domain boundaries utilizing 1D sequence features and predicted 2D inter-residue distance map as input. The 1D features contain the evolutionary and physicochemical information of protein sequences, whereas the 2D distance map includes the structural information of proteins that was rarely used in domain boundary prediction before. The 1D and 2D features are processed by the 1D and 2D U-Nets respectively to generate hidden features. The hidden features are then used by the multi-head attention to predict the probability of each residue of a protein being in a domain boundary, leveraging both local and global information in the features. The residue-level domain boundary predictions can be used to classify proteins as single-domain or multi-domain proteins. It classifies the CASP14 single-domain and multi-domain targets at the accuracy of 75.9%, 13.28% more accurate than the state-of-the-art method. Tested on the CASP14 multi-domain protein targets with expert annotated domain boundaries, the average per-target F1 measure score of the domain boundary prediction by DistDom is 0.263, 29.56% higher than the state-of-the-art method.
BibTeX
@article{mahmud2022multi,
  author = {Sajid Mahmud and Zhiye Guo and Farhan Quadir and Jian Liu and Jianlin Cheng},
  title = {{Multi-head attention-based U-Nets for predicting protein domain boundaries using 1D sequence features and 2D distance maps}},
  journal = {BMC Bioinformatics},
  year = {2022},
  url = {https://openreview.net/forum?id=q2yupYXsha},
}
Figure