TL;DR
This paper introduces ProSSL, a novel deep learning model that leverages protein secondary structure information to detect distant protein homologies. By encoding secondary structure patterns, ProSSL achieves high accuracy in identifying related proteins, even those missed by traditional sequence-based methods, and offers interpretable insights into its predictions.
What Problem Does This Paper Address?
Understanding protein function is crucial for biology, and a primary method for doing so is identifying homologous proteins—those with shared ancestry and likely similar functions. Traditional methods rely on comparing amino acid sequences, but these can fail when proteins have diverged significantly over evolutionary time, accumulating many mutations. While 3D protein structures are more conserved and could help identify distant homologs, obtaining and processing this data is computationally expensive and limited in availability. This paper addresses the challenge of accurately detecting distant protein homologies by proposing a method that bridges the gap between sequence-based and structure-based approaches, using the more readily available and conserved secondary structure information.
Methods
The core of this work is the Protein Secondary Structure Language (ProSSL) model, built upon the Transformer architecture, a standard in large language models. Instead of using amino acid sequences directly, ProSSL takes tokenized representations of protein secondary structures (e.g., helix, strand, coil) as input. The model is trained using a pre-train/fine-tune schema.
- Pre-training: The model is first pre-trained on a large dataset of bacterial protein secondary structures using a masked language modeling (MLM) objective. This is analogous to how language models learn grammar and context from text. The model learns to predict masked secondary structure tokens based on their surrounding context.
- Fine-tuning: After pre-training, the model is fine-tuned on specific protein families (large terminase and portal proteins in this study) for classification tasks.
- Interpretation: The authors utilize Shapley Additive exPlanations (SHAP) values to interpret the model's predictions, allowing them to identify which regions of the secondary structure are most influential in determining homology.
Datasets
- Pre-training: 334,881 bacterial protein sequences from the Swiss-Prot database were used. After filtering, 200,000 sequences were used for training and 134,881 for validation.
- Fine-tuning:
- Terminase prediction: 11,002 large terminase (TerL) sequences and 49,836 non-terminase sequences.
- Portal protein prediction: 14,697 portal protein sequences and 49,827 non-portal protein sequences (derived from the non-terminase dataset).
- Evaluation: The models were evaluated on curated datasets with various phylogenetic splits to ensure robustness.
Key Insights and Ideas
- Secondary structure as a "language": The paper posits that protein secondary structure, like amino acid sequences, can be viewed as a "language" with its own syntax (arrangement of structural elements) that carries functional and evolutionary information.
- Bridging sequence and structure: ProSSL effectively leverages secondary structure, which is more conserved than primary sequence but more readily available and predictable than tertiary structure, offering a powerful intermediate for homology detection.
- Detecting "hidden" homologies: The model demonstrates a significant ability to identify distant homologs that are missed by traditional sequence-based methods like BLASTP, highlighting its utility for uncovering evolutionary relationships.
- Interpretable predictions: The use of SHAP values allows researchers to understand why ProSSL makes a particular classification, pinpointing key secondary structure regions responsible for the decision, which can lead to functional insights.
Key Results
- High Accuracy: Fine-tuned ProSSL models achieved average AUC scores of 98.71-99.96% for terminase prediction and 98.62-99.98% for portal protein prediction, comparable to or exceeding state-of-the-art primary sequence-based models like PROST.
- Efficiency: ProSSL achieved comparable results using significantly less data (three orders of magnitude less) than some baseline models, making it "lightweight" and easier to fine-tune.
- Identification of Novel Homologs: ProSSL successfully identified members of viral protein families that were missed by traditional amino acid similarity searches.
- Structural Insights: SHAP value analysis revealed that ProSSL utilizes information across the entire protein for true positive matches, and for false positives, it can highlight structurally aligned regions that might indicate previously unrecognized homologies. For instance, it identified a potential portal protein based on structural similarity despite lacking sequence similarity.
Limitations and Future Work
- Model Specificity: SHAP values are model-specific, and the authors suggest that future work should focus on developing model-agnostic interpretation metrics for broader comparison.
- Domain-based Analysis: The current model is trained on full-length proteins. The authors suggest that training and fine-tuning on protein domains would expand its applicability to complex eukaryotic proteins with multiple domains.
- Broader Applications: While demonstrated for homology detection, the authors believe ProSSL's ability to learn from secondary structure could benefit other tasks like signal peptide prediction.
- Data Availability: While secondary structure is more accessible than tertiary structure, the availability of high-quality secondary structure predictions for all proteins remains a factor.
Summary updated