TL;DR
This paper introduces GALA, a novel graph transformer model that excels at predicting protein functions, even for proteins with sequences significantly different from those seen during training. By combining adversarial learning with label embedding alignment and leveraging predicted protein structures, GALA achieves remarkable generalization capabilities and biological interpretability.
What Problem Does This Paper Address?
The vast amount of protein sequence data generated by high-throughput sequencing outpaces the experimental determination of their functions. While deep learning methods have shown promise, they often struggle to generalize to novel, non-homologous proteins that differ significantly from the training data. This limitation hinders our ability to annotate the function of newly discovered proteins. Existing methods often rely heavily on sequence similarity, which is insufficient when dealing with divergent sequences. The challenge is to develop models that can accurately predict protein function across a wide range of protein families, even when there's little to no sequence similarity to known proteins.
Methods
GALA employs a multi-pronged approach to tackle protein function prediction:
- Graph Transformer for Representation Learning: It first uses a Graph Convolutional Network (GCN) to capture local structural information. This is followed by a multi-head meta-nodes graph transformer that uses attention mechanisms to aggregate information from neighboring residues and meta-nodes, creating rich protein-level representations.
- Adversarial Learning for Domain Alignment: To ensure domain invariance (i.e., the model's performance doesn't degrade when encountering data from a different distribution), GALA incorporates a domain discriminator. This discriminator is trained adversarially against the main model to minimize the discrepancy between source (training) and target (testing) domains in the learned embedding space.
- Label Embedding Alignment: To improve the discriminability of protein representations, GALA generates label embeddings in a latent space and aligns them with the protein representations. This helps ensure that the learned embeddings effectively capture the functional labels.
Datasets
The study utilizes two primary datasets:
- PDBch: This dataset consists of experimentally solved protein structures from the PDB database, with associated Gene Ontology (GO) terms. To ensure dissimilarity between training and testing sets, MMseqs clustering with a 30% sequence identity threshold was used, resulting in training, validation, and test sets with less than 30% sequence identity between them.
- AFch: This dataset comprises protein structures predicted by AlphaFold2, derived from proteins with low-frequency GO terms from the PDBch dataset. This dataset is crucial for evaluating the model's performance on structures not experimentally determined.
The authors also created a "newly annotated test set" from Swiss-Prot proteins annotated between January 2021 and June 2024 to assess temporal robustness.
Key Insights and Ideas
- Domain Adaptation is Crucial: The paper strongly emphasizes that traditional deep learning models often fail when applied to proteins outside their training distribution. GALA's adversarial domain adaptation is key to overcoming this.
- Structure Matters, Even Predicted Structure: Incorporating AlphaFold2-predicted structures significantly boosts performance, highlighting the value of structural information even when experimental structures are unavailable.
- Generalization Beyond Sequence: GALA's success in predicting functions for proteins with low sequence identity demonstrates that it learns more robust representations than purely sequence-based methods.
- Biological Interpretability: The use of Grad-CAM allows GALA to identify key residues responsible for specific functional annotations, providing valuable biological insights and validating the model's learned representations.
Key Results
- Superior Performance: GALA consistently outperformed several state-of-the-art methods (Blast, DeepGOPlus, TALE+, DeepFRI, Struct2GO, HEAL) across multiple evaluation metrics (AUPR, Fmax, Smin, MCC) on the PDBch test set.
- Robustness on Dissimilar Sequences: GALA showed exceptional performance on the nonhomologous PDBch test set, indicating its ability to generalize to proteins with very different sequences.
- Effectiveness with Predicted Structures: On the AFch test set (using AlphaFold2-predicted structures), GALA significantly outperformed other methods, especially when trained with AFch data.
- Temporal Robustness: GALA demonstrated strong performance on a newly annotated test set, suggesting its robustness over time.
- Interpretability: Grad-CAM analysis successfully identified key functional residues that often corresponded to experimentally verified binding sites.
Limitations and Future Work
- Hierarchical GO Term Structure: The authors plan to incorporate the hierarchical structure of GO terms to further refine the training process, as many current methods require extra postprocessing steps to handle this.
- Integration of Protein-Protein Interactions (PPIs): The paper suggests that integrating PPI data could provide richer information for protein function prediction, enhancing overall robustness and generalizability.
- Further Exploration of Specificity: While GALA performs well across different GO term specificities, further investigation into optimizing performance for very rare GO terms could be beneficial.
Summary updated