One protein is all you need

Anton Bushuiev, Roman Bushuiev, Olga Pimenova, Nikola Zadorozhny, Raman Samusevich, Elisabet Manaskova, Rachel Seongeun Kim, Hannes Stärk, Jiri Sedlar, Martin Steinegger, Tomáš Pluskal, Josef Sivic

TL;DR

ProteinTTT introduces a "test-time training" approach to biology, allowing researchers to customize large protein language models to a single sequence of interest on the fly. By minimizing a model's "surprise" (perplexity) at a specific protein, this method significantly boosts the accuracy of structure, fitness, and function predictions without requiring any additional experimental data.

What Problem Does This Paper Address?

Machine learning models for proteins, like ESMFold or AlphaFold, are typically optimized to perform well on average across massive datasets. However, biologists often care about specific, rare, or highly mutated proteins that might be poorly represented in those training sets. When a model encounters such "out-of-distribution" proteins, its performance often degrades.

The challenge is bridging the gap between broad, dataset-wide optimization and the precision needed for individual proteins of practical interest. Traditionally, improving a model for a specific protein family required collecting more data and performing expensive fine-tuning. ProteinTTT solves this by enabling self-supervised customization using only the single target sequence (or its multiple sequence alignment) at the moment of prediction.

Methods

ProteinTTT leverages the common "Y-shaped" architecture of protein models: a large backbone (feature extractor) followed by a pre-training head (for self-supervision) and a task-specific head (for structure, fitness, etc.).

The Y-shaped paradigm: a backbone shared between self-supervised pre-training and task-specific fine-tuning.

The core innovation is Test-Time Training (TTT):

  1. Customization: When a new protein sequence is provided at test time, the model's backbone is updated for a few steps (e.g., 30 steps) using the same self-supervised objective used during its original pre-training—typically Masked Language Modeling (MLM).
  2. Perplexity Minimization: By training on the target sequence, the model reduces its "perplexity," essentially becoming less "surprised" by the unique patterns of that specific protein.
  3. Efficiency: To make this feasible for massive models (like the 3-billion parameter ESM2), the authors use Low-Rank Adaptation (LoRA), which only updates a tiny fraction of the model's weights, and Stochastic Gradient Descent (SGD) for stability.
  4. Selection: The authors use a confidence metric (like pLDDT for structure prediction) to select the optimal customization step, preventing the model from overfitting to the single sequence.
Overview of the ProteinTTT workflow: from input sequence to customized model and final prediction.

Datasets

The authors evaluated ProteinTTT across several established benchmarks:

  • Structure Prediction: The CAMEO dataset (focusing on low-confidence targets) and CASP14 targets.
  • Fitness Prediction: ProteinGym (2.5 million mutations) and a newly mined dataset from MaveDB for independent validation.
  • Function Prediction: Substrate classification for terpene synthases (TPS) and subcellular localization (DeepLoc).
  • Case Studies: The SAbDab database for antibody-antigen modeling and the Big Fantastic Virus Database (BFVD) containing over 350,000 viral protein structures.

Key Insights and Ideas

  • Perplexity as a Proxy for Quality: A central "aha moment" is the strong correlation between a model's perplexity and its downstream accuracy. As the model "understands" the sequence better through TTT, its structural predictions become more physically plausible.
  • Customization is Universal: The method works across diverse architectures, including bidirectional (ESM2), autoregressive (ProGen2), and discrete diffusion models (DPLM2).
  • One Sequence is Enough: Remarkably, customizing to just one sequence can be more effective than standard methods that rely on large multiple sequence alignments (MSAs), especially for proteins with few known homologs (like viral proteins).
  • Iterative Refinement: Figure 3 illustrates how structure prediction quality (TM-score) improves step-by-step as perplexity drops during the TTT process.
Iterative improvement of structure prediction quality as perplexity decreases during customization.

Key Results

  • Structure Prediction: ProteinTTT improved the TM-score of challenging targets significantly. In one dramatic case (CASP14 target T1074), it refined a failed prediction (TM-score 0.63) into a high-quality one (TM-score 0.84).
  • Fitness Prediction: The combination of ProSST + ProteinTTT achieved a new state-of-the-art on the ProteinGym benchmark.
  • Viral Proteins: In the Big Fantastic Virus Database, ProteinTTT improved 19% of the structures, providing high-quality predictions where both AlphaFold2 and ESMFold previously struggled.
  • Antibodies: The method significantly improved the modeling of highly variable CDR loops, which are critical for understanding how antibodies bind to antigens.
Figure 1: Example of protein structure predic- tion after single-protein model customization via ProteinTTT. ESMFold poorly predicts the structure of the CASP14 target T1074 (white) be- cause the underlying language model ESM2 poorly fits the sequence, as indicated by the high perplex- ity (left and Fig. 2E in Lin et al. (2023)). Self- supervised test-time customization of ESM2 to the single sequence of T1074 reduces the perplexity, resulting in improved structure prediction (right).

Limitations and Future Work

  • Model Scope: Currently, the method is designed for Transformer-based Protein Language Models (PLMs). Extending this to other architectures like Evoformer (used in AlphaFold) is a promising next step.
  • Confidence Metrics: While pLDDT works well for structure, reliable confidence metrics for fitness and function are still lacking. Developing task-agnostic, unsupervised confidence measures would help determine the optimal number of TTT steps for all tasks.
  • Failure Modes: In a small number of cases, customization can slightly decrease performance. Understanding these failure modes—often linked to ambiguities in the ground-truth data—remains an open research question.

Summary updated

Abstract

Generalization beyond training data remains a central challenge in machine learning for biology. A common way to enhance generalization is self-supervised pre-training on large datasets. However, aiming to perform well on all possible proteins can limit a model's capacity to excel on any specific one, whereas experimentalists typically need accurate predictions for individual proteins they study, often not covered in training data. To address this limitation, we propose a method that enables self-supervised customization of protein language models to one target protein at a time, on the fly, and without assuming any additional data. We show that our Protein Test-Time Training (ProteinTTT) method consistently enhances generalization across different models, their sizes, and datasets. ProteinTTT improves structure prediction for challenging targets, achieves new state-of-the-art results on protein fitness prediction, and enhances function prediction on two tasks. Through two challenging case studies, we also show that customization via ProteinTTT achieves more accurate antibody-antigen loop modeling and enhances 19% of structures in the Big Fantastic Virus Database, delivering improved predictions where general-purpose AlphaFold2 and ESMFold struggle.
BibTeX
@inproceedings{bushuiev2026one,
  author = {Anton Bushuiev and Roman Bushuiev and Olga Pimenova and Nikola Zadorozhny and Raman Samusevich and Elisabet Manaskova and Rachel Seongeun Kim and Hannes Stärk and Jiri Sedlar and Martin Steinegger and Tomáš Pluskal and Josef Sivic},
  title = {{One protein is all you need}},
  booktitle = {International Conference on Learning Representations},
  year = {2026},
  url = {https://openreview.net/forum?id=5zAde2jch7},
}
Figure