ProtTox: Toxin identification from Protein Sequences

Sathappan Muthiah, Debanjan Datta, Mohammad Raihanul Islam, P. J. Butler, Andrew Warren, Naren RamakrishnanbioRxiv 2020 · DOI

TL;DR

This paper introduces ProtTox, a novel deep learning approach for identifying toxic proteins from their amino acid sequences. ProtTox achieves a significant improvement in performance over existing methods, offering a more efficient and accurate way to flag potentially harmful proteins for applications in healthcare and synthetic biology.

What Problem Does This Paper Address?

The vast and ever-growing number of protein sequences generated by modern sequencing technologies presents a significant challenge for manual annotation of their functions. Identifying toxic proteins is particularly crucial for healthcare (e.g., understanding disease mechanisms, developing antidotes) and synthetic biology (e.g., designing safe biological systems). Traditional methods for protein function annotation are often expensive and time-consuming. Existing computational approaches struggle with challenges like protein homology (similar sequences having different functions), multi-functionality, and structural diversity. This paper aims to bridge this gap by developing an automated, machine learning-based method for accurate toxin identification directly from protein sequences.

Methods

ProtTox is a deep learning model that leverages a supervised approach to learn from existing annotated protein sequences. The core architecture consists of three main components:

  1. Language Model (LM): A pre-trained bidirectional LSTM (Long Short-Term Memory) network that learns contextual information from the amino acid sequence. This LM is kept fixed during training.
  2. BiLSTM Encoder (E): Another bidirectional LSTM network that takes the output of the LM and an embedding vector for each amino acid as input. It processes this information to generate a representation for each position in the sequence.
  3. Aggregation (A) and Classification Layer (C): The representations from the encoder are aggregated (specifically, using mean reduction) to form a single sequence representation. This representation is then passed through a dense layer with a sigmoid activation to predict the probability of the protein being a toxin.

An extension, ProtTox-Attn, incorporates an attention mechanism. This mechanism allows the model to focus on specific parts of the sequence that are more indicative of toxicity, potentially improving performance by not treating all parts of the sequence equally. This attention mechanism uses dot-product attention with a sigmoid activation and is trained with an additional fused LASSO term to encourage sparsity and smoothness.

Datasets

The models were trained using positive examples from the Swiss-Prot database (keyword "KW-0800") and toxins from T3DB and VFDB. Negative examples were sourced from the ToxClassifier dataset, which includes non-toxic sequences from Swiss-Prot and "hard negative" examples identified via BLASTp searches against toxin-like proteins with physiological functions.

For testing, a random held-out sample of 1000 sequences with the Uniprot keyword "KW-0800" was used as the positive set, and 1000 sequences from PATRIC's metabolically essential genes list served as the negative set.

Key Insights and Ideas

  • Deep Learning for Toxin Identification: The paper demonstrates that deep learning models, particularly those employing recurrent neural networks like LSTMs, are highly effective for the challenging task of toxin identification from protein sequences.
  • Attention Mechanism Enhances Performance: The ProtTox-Attn model, which incorporates an attention mechanism, shows improved performance, suggesting that focusing on salient regions of a protein sequence can be crucial for accurate toxicity prediction.
  • Sub-type Prediction Aids General Toxin Prediction: Breaking down the broad "toxin" class into specific sub-types (e.g., neurotoxin, enterotoxin) and training the model to predict these sub-types, in addition to the general toxin class, significantly improves the overall toxin prediction capability of ProtTox-Attn. This suggests that learning finer-grained distinctions helps the model build more robust representations.
  • Sequence Length Matters: The performance of models can be sensitive to sequence length. While ProtTox-Attn performs well on longer sequences, the optimal F1-score for other models (like SWEM) was observed around a sequence length of 500, indicating a trade-off between precision and recall with increasing length.

Key Results

  • ProtTox achieved an F1-score of 0.812 for toxin prediction, which was approximately 5% higher than the closest performing baseline.
  • ProtTox-Attn achieved a near-perfect score of 99.8% for non-toxin prediction.
  • When trained for sub-type prediction, ProtTox-Attn showed improved overall toxin prediction capability, achieving an F1-score of 0.812, which was 5% better than the best SWEM model.
  • The paper highlights that while ProtTox models might have lower recall compared to SWEM in some cases, they achieve better precision for non-toxin prediction, demonstrating their utility for vanilla toxin prediction.

Limitations and Future Work

  • Sequence Length Sensitivity: While ProtTox-Attn performs well on longer sequences, the optimal sequence length for other models can vary, suggesting that sequence length truncation strategies might be necessary for some approaches.
  • Model Complexity: The paper notes that simpler models like SWEM perform poorly when the task complexity increases (e.g., predicting sub-types), suggesting that deeper, more complex models are better suited for intricate classification tasks.
  • Future Directions: The authors suggest several avenues for future work, including:
  • Incorporating richer domain knowledge, such as Protein-Protein Interaction (PPI) data and 3D protein structures.
  • Extending the model to handle other protein classification tasks.
  • Further exploring the relationship between sequence length and model performance.

Summary updated

Abstract

Abstract Toxin classification of protein sequences is a challenging task with real world applications in healthcare and synthetic biology. Due to an ever expanding database of proteins and the inordinate cost of manual annotation, automated machine learning based approaches are crucial. Approaches need to overcome challenges of homology, multi-functionality, and structural diversity among proteins in this task. We propose a novel deep learning based method ProtTox, that aims to address some of the shortcomings of previous approaches in classifying proteins as toxins or not. Our method achieves a performance of 0.812 F1-score which is about 5% higher than the closest performing baseline.
BibTeX
@misc{muthiah2020prottox,
  author = {Sathappan Muthiah and Debanjan Datta and Mohammad Raihanul Islam and P. J. Butler and Andrew Warren and Naren Ramakrishnan},
  title = {{ProtTox: Toxin identification from Protein Sequences}},
  year = {2020},
  doi = {10.1101/2020.04.18.048439},
  howpublished = {bioRxiv},
}
Figure