Restoring data balance via generative models of T cell receptors for antigen-binding prediction

Emanuele Loffredo, Mauro Pastore, Simona Cocco, Rémi Monasson

Abstract

Unveiling specificity in T cell recognition of antigens represents a major step to understand the immune system response. Many supervised machine learning approaches have been designed to build sequence-based predictive models of such specificity using binding and non-binding receptor-antigen data. Due to the scarcity of known specific T cell receptors for each antigen compared to the abundance of non-specific ones, available datasets are heavily imbalanced and make the goal of achieving solid predictive performances very challenging. Here, we propose to restore data balance through data augmentation using generative unsupervised models. We then use these augmented data to train supervised models for prediction of peptide-specific T cell receptors, or binding pairs of peptide and T cell receptor sequences. We show that our pipeline yields increased performance in prediction tasks of T cell receptors specificity. More broadly, our pipeline provides a general framework that could be used to restore balance in other computational problems involving biological sequence data.

Introduction

Predicting whether a T-cell receptor (TCR) will bind to a specific viral or bacterial peptide is a fundamental challenge in computational immunology. Success in this area would revolutionize vaccine design and personalized immunotherapy. However, the field is hindered by a severe data imbalance: for any given pathogen, the number of confirmed binding TCRs in public databases is dwarfed by the millions of non-binding sequences. Standard supervised machine learning models trained on such lopsided data often fail to generalize, becoming biased toward the majority "non-binder" class.

Methodology: A Generative Pipeline for Data Balancing

To address this, the authors developed a two-step pipeline that restores balance to training datasets through intelligent data augmentation. Unlike simple oversampling (which merely duplicates existing data), this approach uses unsupervised generative models—specifically Restricted Boltzmann Machines (RBMs) and BERT-like Transformers—to learn the underlying "grammar" of known binding sequences.

The proposed pipeline for restoring data balance using generative models before supervised classification.

By sampling from these models, the researchers "hallucinate" new, high-quality synthetic binding sequences that capture complex, high-order correlations between amino acids. These synthetic sequences are then combined with real data to train a 1D Convolutional Neural Network (CNN) for either peptide-specific or pan-specific binding prediction.

Results and Performance Gains

The generative augmentation strategy significantly outperformed traditional methods. For epitopes with very few known binders—the most challenging scenarios—the pipeline yielded up to a 20% increase in balanced accuracy compared to simple undersampling of the majority class.

Performance comparison (ACC) for various epitope sets, highlighting the effectiveness of BERT and RBM-based augmentation.

Both the physics-based RBM and the NLP-based BERT architectures proved effective, demonstrating the framework's versatility. Furthermore, in pan-specific models, augmenting data for one epitope was found to improve performance on other, chemically similar epitopes, suggesting that the CNN learns generalizable features of TCR-peptide interactions.

Geometric Insights and Future Directions

Using a synthetic "Lattice Protein" model, the authors visualized how data imbalance physically shifts the decision boundary in the model's feature space. Generative augmentation effectively pushes this boundary back to its optimal position, enabling cleaner separation between binders and non-binders.

While the current model focuses on the CDR3β chain, future work aims to incorporate alpha chains and structural context to further bridge the "unseen epitope gap." Despite current computational costs, this generative framework offers a robust solution for restoring balance in biological sequence datasets beyond immunology.

Summary updated

Abstract

Unveiling specificity in T cell recognition of antigens represents a major step to understand the immune system response. Many supervised machine learning approaches have been designed to build sequence-based predictive models of such specificity using binding and non-binding receptor-antigen data. Due to the scarcity of known specific T cell receptors for each antigen compared to the abundance of non-specific ones, available datasets are heavily imbalanced and make the goal of achieving solid predictive performances very challenging. Here, we propose to restore data balance through data augmentation using generative unsupervised models. We then use these augmented data to train supervised models for prediction of peptide-specific T cell receptors, or binding pairs of peptide and T cell receptor sequences. We show that our pipeline yields increased performance in prediction tasks of T cell receptors specificity. More broadly, our pipeline provides a general framework that could be used to restore balance in other computational problems involving biological sequence data.
BibTeX
@misc{loffredo2025restoring,
  author = {Emanuele Loffredo and Mauro Pastore and Simona Cocco and Rémi Monasson},
  title = {{Restoring data balance via generative models of T cell receptors for antigen-binding prediction}},
  year = {2025},
  url = {https://openreview.net/forum?id=Q10yQ6lQbY},
  howpublished = {OpenReview},
}
Figure