Evaluating the advancements in protein language models for encoding strategies in protein function prediction: a comprehensive review

Jiaying Chen, Jingfu Wang, Yue Hu, Xia Li, Yurong Qian, Chenxi SongFrontiers in Bioengineering and Biotechnology 2025 · DOI

Abstract

Protein function prediction is crucial in several key areas such as bioinformatics and drug design. With the rapid progress of deep learning technology, applying protein language models has become a research focus. These models utilize the increasing amount of large-scale protein sequence data to deeply mine its intrinsic semantic information, which can effectively improve the accuracy of protein function prediction. This review comprehensively combines the current status of applying the latest protein language models in protein function prediction. It provides an exhaustive performance comparison with traditional prediction methods. Through the in-depth analysis of experimental results, the significant advantages of protein language models in enhancing the accuracy and depth of protein function prediction tasks are fully demonstrated.

Introduction

The biological community currently faces a massive "annotation gap." As of early 2024, the UniProt database contains over 240 million protein sequences, yet less than 0.3% have been experimentally validated. Traditional laboratory verification is too slow and resource-intensive to keep pace with the exponential growth of genomic data.

Historically, researchers relied on homology-based methods like BLAST, which assume that similar sequences share similar functions. However, these methods often fail in the "twilight zone," where proteins have no close relatives. While standard deep learning approaches (CNNs and RNNs) offered improvements, they struggled with long-range dependencies and required large amounts of labeled data. Protein Language Models (PLMs) address these issues by treating amino acid sequences as a biological language, learning the "grammar" of evolution from massive unlabeled datasets.

Methods and Model Architectures

This review benchmarks nine modern PLMs, categorized by their architecture and training objectives. These models generally follow a "feature extraction" strategy where a pre-trained encoder generates high-dimensional embeddings for a sequence, which are then used to predict Gene Ontology (GO) terms.

Architecture of a Protein Language Model for function prediction

The models evaluated include:

  • BERT-style (Masked Language Modeling): Models such as ESM-1b, ESM2, and ProtBERT use a "fill-in-the-blank" approach. By hiding specific amino acids during pre-training, the model learns to predict them based on context, capturing the latent semantics of protein sequences.
  • T5-style (Encoder-Decoder): Models like ProtT5 and Ankh treat protein tasks as a translation problem, mapping sequences to functional descriptions.
Comparison of modern Protein Language Models and their architectures

The study utilizes primary benchmarks including CAFA3, CAFA4, and a specific Human2024 dataset to ensure reliability across species. Evaluation is conducted across three sub-ontologies: Biological Process (BP), Molecular Function (MF), and Cellular Component (CC).

Key Insights and Results

The analysis reveals several critical advantages of PLMs over traditional methods:

  • Evolutionary Grammar: PLMs capture "latent semantics," understanding which amino acid substitutions are biologically permissible. This allows them to succeed even when sequence identity is low.
  • Superior Performance: The ESM family consistently outperformed other models. ESM-1b achieved a 10% improvement in Fmax over previous state-of-the-art deep learning models. On the CAFA4 dataset, it reached an Fmax of 0.736 for Cellular Component prediction.
  • Efficiency and Scale: While larger models generally perform better, the ESM2 650M parameter model often rivaled the 3B version, suggesting an optimal balance between model size and predictive accuracy.
  • Localized Feature Sensitivity: PLMs are particularly effective at Molecular Function prediction, likely due to their sensitivity to localized sequence motifs like binding sites.

Conclusion and Future Directions

While PLMs represent a significant leap forward, challenges remain. The "black box" nature of these models makes interpreting specific functional assignments difficult. Furthermore, current models primarily focus on single sequences, missing the broader context of Protein-Protein Interaction (PPI) networks and Post-Translational Modifications (PTMs). Future advancements may involve integrating 3D structural data, such as contact maps from AlphaFold, to better capture how distant sequence elements interact in three-dimensional space to define function.

Summary updated

Abstract

Protein function prediction is crucial in several key areas such as bioinformatics and drug design. With the rapid progress of deep learning technology, applying protein language models has become a research focus. These models utilize the increasing amount of large-scale protein sequence data to deeply mine its intrinsic semantic information, which can effectively improve the accuracy of protein function prediction. This review comprehensively combines the current status of applying the latest protein language models in protein function prediction. It provides an exhaustive performance comparison with traditional prediction methods. Through the in-depth analysis of experimental results, the significant advantages of protein language models in enhancing the accuracy and depth of protein function prediction tasks are fully demonstrated.
BibTeX
@inproceedings{chen2025evaluating,
  author = {Jiaying Chen and Jingfu Wang and Yue Hu and Xia Li and Yurong Qian and Chenxi Song},
  title = {{Evaluating the advancements in protein language models for encoding strategies in protein function prediction: a comprehensive review}},
  booktitle = {Frontiers in Bioengineering and Biotechnology},
  year = {2025},
  doi = {10.3389/fbioe.2025.1506508},
}
Figure