Data augmentation enables label-specific generation of homologous protein sequences

Lorenzo Rosset, Martin Weigt, Francesco ZamponiarXiv 2025 · arXiv

Abstract

Accurately annotating and controlling protein function from sequence data remains a major challenge, particularly within homologous families where annotated sequences are scarce and structural variation is minimal. We present a two-stage approach for semi-supervised functional annotation and conditional sequence generation in protein families using representation learning. First, we demonstrate that protein language models, pretrained on large and diverse sequence datasets and possibly finetuned via contrastive learning, provide embeddings that robustly capture fine-grained functional specificities, even with limited labeled data. Second, we use the inferred annotations to train a generative probabilistic model, an annotation-aware Restricted Boltzmann Machine, capable of producing synthetic sequences with prescribed functional labels. Across several protein families, we show that this approach achieves highly accurate annotation quality and supports the generation of functionally coherent sequences. Our findings underscore the power of combining self-supervised learning with light supervision to overcome data scarcity in protein function prediction and design.

Introduction

In the post-genomic era, the gap between the number of known protein sequences and those with experimentally verified functions continues to widen. While structural prediction tools like AlphaFold have made significant strides, predicting specific biological functions remains difficult. This is particularly true for homologous protein families, where sequences share high structural similarity but perform distinct tasks. The primary obstacle is data scarcity: in databases like UniProt, only approximately 0.2% of sequences possess human-curated functional annotations. Traditional machine learning methods often struggle in this regime, frequently relying on simple sequence similarity rather than capturing the underlying functional signals.

Methodology: A Two-Stage Semi-Supervised Pipeline

To address the challenge of functional annotation and design, we propose a two-stage framework that leverages the latent representations learned by protein language models (pLMs).

Stage 1: Data Augmentation (The Predictor)

We utilize a pre-trained pLM (such as ProtBERT) to generate high-dimensional embeddings for a protein family. These models, trained on millions of diverse sequences, capture the "grammar" of proteins. Even without task-specific fine-tuning, these embeddings often cluster by function. By training a simple logistic regression classifier on a small set of labeled sequences (as few as 100), we can accurately predict functional labels for the remaining thousands of unlabeled sequences in the family. To ensure the model learns functional features rather than phylogenetic biases, we employ a rigorous data-splitting strategy based on sequence identity thresholds ($t_1$).

Stage 2: Conditional Generation (The Generator)

In the second stage, the newly annotated sequences are used to train an annotation-aware Restricted Boltzmann Machine (RBM). This generative probabilistic model learns the joint distribution of protein sequences and their corresponding functional labels. Once trained, the RBM can be sampled conditionally to generate novel, synthetic sequences tailored to a specific functional prescription.

Overview of the two-stage pipeline for functional annotation and conditional sequence generation.

Experimental Validation and Results

The framework was validated across four diverse protein families: Response Regulators (RR), SH3 domains, the Globin family, and Chorismate Mutase (CM).

Annotation Accuracy and Robustness

A key finding is the robustness of pLM-based embeddings compared to traditional methods. Standard sequence-alignment methods (MSA-based) show a significant performance drop when test sequences share less than 40% identity with the training set. In contrast, our pLM-based predictor maintained an Area Under the Curve (AUC) of 0.96 on the Response Regulator dataset, even at low sequence identity thresholds. This suggests that foundation models capture high-level functional features that transcend simple primary sequence similarity.

Performance metrics showing high annotation accuracy and robustness to low sequence identity.

Generative Fidelity

The sequences generated by the label-aware RBM were found to be statistically indistinguishable from natural sequences. They successfully recovered complex amino acid correlations and functional motifs. Furthermore, a "self-consistency" test—where generated sequences were re-labeled by the predictor—showed high agreement (AUC $\simeq$ 0.89), confirming that the generator effectively learned the mapping between sequence space and functional labels.

Discussion and Future Work

Our results demonstrate that "foundation" protein language models are inherently "function-ready," providing a powerful starting point for semi-supervised learning. This pipeline allows researchers to bootstrap functional labels from minimal data, enabling the design of specialized proteins for specific biological tasks.

While these computational results are promising, future work will focus on in vitro experimental validation to confirm the biological activity of the generated sequences. Additionally, exploring larger foundation models like ESM2 may further improve the resolution of extremely subtle functional differences. This approach opens new avenues for exploring evolutionary trajectories and engineering proteins with novel or optimized functions.

Summary updated

Abstract

Accurately annotating and controlling protein function from sequence data remains a major challenge, particularly within homologous families where annotated sequences are scarce and structural variation is minimal. We present a two-stage approach for semi-supervised functional annotation and conditional sequence generation in protein families using representation learning. First, we demonstrate that protein language models, pretrained on large and diverse sequence datasets and possibly finetuned via contrastive learning, provide embeddings that robustly capture fine-grained functional specificities, even with limited labeled data. Second, we use the inferred annotations to train a generative probabilistic model, an annotation-aware Restricted Boltzmann Machine, capable of producing synthetic sequences with prescribed functional labels. Across several protein families, we show that this approach achieves highly accurate annotation quality and supports the generation of functionally coherent sequences. Our findings underscore the power of combining self-supervised learning with light supervision to overcome data scarcity in protein function prediction and design.
BibTeX
@misc{rosset2025data,
  author = {Lorenzo Rosset and Martin Weigt and Francesco Zamponi},
  title = {{Data augmentation enables label-specific generation of homologous protein sequences}},
  year = {2025},
  eprint = {2507.15651},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2507.15651},
  howpublished = {arXiv},
}
Figure