Partial order relation-based gene ontology embedding improves protein function prediction

Wenjing Li, Bin Wang, Jin Dai, Yan Kou, Xiaojun Chen, Yi Pan, Shuangwei Hu, Zhenjiang Zech XuBriefings in Bioinformatics 2024 · DOI

TL;DR

This paper introduces PO2Vec, a novel method for learning gene ontology (GO) term embeddings by leveraging partial order relationships within the GO hierarchy. This improved representation learning leads to a new protein function prediction model, PO2GO, which demonstrates superior performance across various biological tasks, including predicting protein functions with higher accuracy and specificity, especially in low-data scenarios.

What Problem Does This Paper Address?

Understanding protein function is crucial for deciphering biological processes. Experimental methods for annotating protein functions are slow and expensive, creating a bottleneck as the volume of sequenced proteins grows. Computational methods, particularly those using deep learning, have shown promise, but they struggle with the complex, hierarchical, and unbalanced nature of the Gene Ontology (GO) – the standard framework for describing protein functions. Existing GO term embedding methods often rely on co-occurrence information but fail to fully capture the intricate topological structure of the GO's directed acyclic graph (DAG). This limits their ability to accurately represent the semantic relationships between GO terms, which is essential for effective protein function prediction.

Methods

The core innovation of this paper is the PO2Vec model for learning GO term embeddings. Unlike previous methods that primarily use ancestral co-occurrence, PO2Vec explicitly incorporates partial order relationships derived from the GO's hierarchical structure. Specifically, it defines Shortest Reachable Path (SRP) lengths between GO terms to quantify their semantic proximity.

The PO2Vec method employs a contrastive learning strategy. It aims to push embeddings of semantically similar GO terms (e.g., parent-child or sibling terms) closer together in the embedding space while pushing dissimilar terms further apart. To achieve this, they introduce a stratified sampling approach for selecting positive and negative samples. Positive samples are chosen from directly or indirectly reachable terms with specific SRP lengths, while negative samples are strategically sampled from different categories of reachability (directly, indirectly, and unreachable) to ensure a balanced representation.

The learned GO embeddings are then integrated into a protein function prediction model called PO2GO. PO2GO uses the ESM-1b protein language model to extract protein sequence embeddings. These protein embeddings and the PO2Vec-generated GO term embeddings are then projected into a common space using multi-layer perceptrons (MLPs). A final MLP layer predicts the probability of each GO term being associated with a given protein.

Datasets

The paper utilizes existing GO datasets for training and evaluation. Specific datasets are not explicitly named in the main text but are referred to as being detailed in the Supplementary Material. For protein function prediction, they use the CAFA3 dataset and the swissprot dataset.

Key Insights and Ideas

  • Partial Order is Key: The paper strongly suggests that capturing the nuanced topological relationships (beyond simple ancestry) within the GO hierarchy, specifically through shortest reachable paths, is critical for high-quality GO term embeddings.
  • Contrastive Learning for Semantics: Applying contrastive learning with carefully sampled positive and negative pairs effectively pushes semantically related GO terms together and unrelated terms apart, leading to more meaningful embeddings.
  • Stratified Sampling for Robustness: The stratified sampling strategy for negative samples is crucial. It prevents the model from being overwhelmed by distant or unrelated terms and ensures that closer relationships are adequately learned, especially for depth prediction.
  • Embeddings Enhance Downstream Tasks: The quality of GO term embeddings directly translates to improved performance in downstream tasks like protein function prediction, demonstrating the practical utility of the learned representations.
  • Few-Shot Learning Advantage: PO2Vec-based embeddings seem to offer a significant advantage in few-shot learning scenarios, where limited training data is available, suggesting that rich pre-trained embeddings can compensate for data scarcity.

Key Results

  • Depth Prediction: PO2Vec significantly outperforms baseline methods in predicting the depth of GO terms, indicating its superior ability to capture hierarchical information.
  • Distinguishability: PO2Vec embeddings better distinguish between ancestor and non-ancestor GO terms compared to several other methods, as measured by 1-Wasserstein distances.
  • Domain Encoding: UMAP visualization shows clear separation of GO terms into their respective domains (BPO, CCO, MFO) when using PO2Vec embeddings.
  • Protein Function Prediction (CAFA3 & Swissprot): PO2GO, powered by PO2Vec, achieves state-of-the-art performance on the CAFA3 and swissprot datasets, outperforming methods like Diamond, DeepGOPlus, ESM-1b&DeepGOA, and ESM-1b&TALE in metrics like Fmax, Smin, and AUPR.
  • Specificity: PO2GO demonstrates higher or comparable specificity in protein annotation, meaning it can predict more specialized GO terms.
  • Few-Shot Performance: PO2GO shows mitigated performance degradation in few-shot scenarios compared to other methods, highlighting its robustness with limited data.

Limitations and Future Work

  • Limited Relationship Types: The current SRP-based partial order constraint primarily considers "is_a" and "part_of" relationships, which constitute about 88% of GO relationships. The authors suggest that incorporating the remaining relationship types could further improve embeddings.
  • Textual Information Ignored: The study focuses on the GO DAG structure. The authors propose that incorporating textual descriptions of GO terms using NLP techniques could provide additional information for richer representation learning.
  • Interchangeable Protein Encoders: While ESM-1b was used, the PO2GO architecture is generic and can accommodate other pre-trained protein language models.
  • Broader Knowledge Graph Applications: The authors suggest that their embedding approach could be extended to represent other knowledge graphs, such as the Human Phenotype Ontology, Disease Ontology, and SNOMED CT.

Summary updated

BibTeX
@article{li2024partial,
  author = {Wenjing Li and Bin Wang and Jin Dai and Yan Kou and Xiaojun Chen and Yi Pan and Shuangwei Hu and Zhenjiang Zech Xu},
  title = {{Partial order relation-based gene ontology embedding improves protein function prediction}},
  journal = {Briefings in Bioinformatics},
  year = {2024},
  doi = {10.1093/bib/bbae077},
}
Figure