Protein Fold Classification at Scale: Benchmarking and Pretraining

Dexiong Chen, Andrei Manolache, Mathias Niepert, Karsten Borgwardt

In biology, a fundamental maxim is that "structure is function." Classifying a protein's fold—its three-dimensional topology—is an essential step in deciphering its biological role. However, the field of computational structural biology has been trapped in a "small data" bottleneck for supervised learning. While tools like AlphaFold have provided millions of predicted structures, existing supervised benchmarks for fold classification remain tiny (often containing fewer than 15,000 proteins) and are plagued by redundant data.

Furthermore, current state-of-the-art protein representation models face a dichotomy: they are either massive language models that incur exorbitant computational costs, or geometric models that fail to scale effectively to the millions of structures now available. To trigger an "ImageNet moment" for protein structural biology, researchers have introduced TEDBench, a massive, high-quality benchmark, alongside Masked Invariant Autoencoders (MiAE), a highly efficient self-supervised model architecture designed to conquer this new scale.

TEDBench: A Massive New Benchmark

TEDBench represents a paradigm shift in scale, offering a dataset 30 times larger than previous benchmarks. It comprises 462,175 high-confidence, non-redundant protein structures derived from the Encyclopedia of Domains (TED) and the AlphaFold Database.

To ensure rigorous evaluation, TEDBench utilizes CATH labels—a gold-standard hierarchical classification for protein folds. Alongside the primary benchmark, the authors curated an external test set of 27,638 experimentally determined structures from CATH v4.4. This allows researchers to test how well models trained on predicted structures generalize to real laboratory data. The pretraining corpus is even larger, encompassing approximately 750,000 unannotated structures.

Masked Invariant Autoencoders (MiAE)

To handle this massive influx of structural data, the authors propose MiAE, an asymmetric encoder-decoder architecture inspired by Masked Autoencoders (MAE) used in computer vision. MiAE introduces several key innovations tailored to protein structures:

  • Aggressive 90% Masking: The model randomly hides up to 90% of the protein's residues (frames).
  • SE(3)-Invariant Encoder: The "heavy" encoder processes only the 10% of visible residues. It is mathematically designed to be invariant to global rotations and translations, meaning the protein's absolute orientation in 3D space does not affect the encoding.
  • Lightweight Decoder: A small transformer takes the encoder's output, combined with "mask tokens," to reconstruct the full 3D coordinates of the protein backbone.
  • Composite Loss: The model is optimized by minimizing errors in pairwise distances and directions between atoms, alongside an "inverse folding" task that predicts the amino acid sequence from the structure.

The 90% Masking Paradox

One of the paper's most fascinating insights is the necessity of extreme masking. In natural language processing, models typically mask 15% of words; in computer vision, about 75% of pixels. Proteins, however, are so structurally redundant that at lower masking ratios, the model "cheats" by simply interpolating from neighboring residues. It is only when 90% of the data is removed that the model is forced to learn true "global reasoning" and understand the overall fold.

Key Results and Efficiency

MiAE demonstrates that massive scale does not require massive inefficiency. By only encoding 10% of the residues, MiAE is incredibly fast.

  • State-of-the-Art Performance: On TEDBench, the MiAE-B model (102M parameters) achieved a 74.56 macro-F1 score. This significantly outperforms much larger models, such as the 650-million-parameter ESM2 (66.19) and SaProt-650M (73.48).
  • Training Speed: Efficiency is where MiAE truly shines. Pretraining MiAE took only 768 GPU hours. In stark contrast, comparable baselines like ESM2-650M required an estimated 138,240 GPU hours—making MiAE roughly 180 times more efficient to train.
  • Scaling Laws: MiAE exhibits clear scaling behavior. As the model size increases from 29M to 339M parameters, or as the amount of pretraining data grows, downstream performance consistently improves.
  • Generalization: Models trained on AlphaFold's predicted structures generalized remarkably well to experimental structures. Performance was often higher on the experimental test set, likely due to the superior quality of human-curated CATH labels.

Limitations and Future Directions

While TEDBench and MiAE establish a strong new recipe for protein fold classification, the authors note a few limitations. Currently, TEDBench assigns a label based on the largest domain in a protein, which may overlook smaller, biologically significant domains in multi-domain proteins. Additionally, while MiAE excels at fold classification, its performance on other critical tasks—such as protein-protein interaction or ligand binding—remains to be explored.

Looking forward, the authors suggest that the field could evolve from simple classification to a "detection" framework. In such a paradigm, a model would simultaneously segment a complex protein into its constituent domains and classify each one, bringing computational structural biology even closer to capturing the full complexity of nature.

Summary updated

Abstract

Classifying protein topology is essential for deciphering biological function, but progress is held back by the lack of large-scale benchmarks that avoid duplicates and by models that do not scale well. We introduce TEDBench, a large-scale, non-redundant benchmark for protein fold classification constructed from the Encyclopedia of Domains (TED) and Foldseek-clustered AlphaFold structures. We show that on TEDBench, current protein representation learning methods either require very large models or fail to deliver strong performance. To address this challenge, we propose Masked Invariant Autoencoders (MiAE), a self-supervised framework for protein structure representation learning. MiAE uses an extremely high masking ratio of up to $90\%$ with an $\mathrm{SE(3)}$-invariant encoder and a lightweight decoder that reconstructs backbone coordinates from the latent representation and mask tokens. MiAE scales well and outperforms supervised counterparts and state-of-the-art baselines on TEDBench, establishing a strong recipe for protein fold classification. To test transfer beyond AlphaFold structures, we further benchmark on a curated dataset from experimental structures of CATH v4.4. TEDBench is available at https://github.com/BorgwardtLab/TEDBench.
BibTeX
@inproceedings{chen2026protein,
  author = {Dexiong Chen and Andrei Manolache and Mathias Niepert and Karsten Borgwardt},
  title = {{Protein Fold Classification at Scale: Benchmarking and Pretraining}},
  booktitle = {International Conference on Machine Learning},
  year = {2026},
  url = {https://openreview.net/forum?id=jPKqiaPTEd},
}
Figure