Multi-Prototype Folding: One Sequence, Several Structures, One Forward Pass

Proteins are not rigid objects. Many adopt two or more distinct stable folds (so-called metamorphic proteins), and nearly all fluctuate among loosely related conformations while performing their function. Yet the fastest structure predictors — the "alignment-free" models that read a single amino-acid sequence and directly output 3D coordinates, without searching sequence databases for evolutionary relatives — return exactly one structure. ESMFold and OmegaFold, the leading systems of this kind, compress a protein's entire conformational repertoire into a single point estimate, a limitation documented directly in a head-to-head comparison by Hyskova et al. (Balancing speed and precision in protein folding: a comparison of AlphaFold2, ESMFold, and OmegaFold). The existing fixes give up what makes these models valuable: generative samplers such as EigenFold and ESMFLOW need 100–300 sequential denoising steps and, by their own authors' admission, produce variance that mostly reflects the model's uncertainty rather than genuine biological flexibility.

This proposal imports an idea from a completely different literature. In computer vision, representing each category by a single vector is known to destroy information when a category is internally diverse; replacing one class centroid with a bank of learned prototypes — multiple vectors per class, each covering one mode of variation — reliably improves fine-grained retrieval (SoftTriple Loss: Deep Metric Learning Without Triplet Sampling) and 3D geometric segmentation (Few-Shot 3D Point Cloud Semantic Segmentation). No work in the collected literature applies this to protein structure prediction. We propose to build it into an alignment-free predictor: a small bank of conformational prototypes inside the folding head, so that one feed-forward pass decodes a handful of discrete candidate structures, each corresponding to an explicit structural basin. The speed of single-sequence prediction is preserved; the single-state bottleneck is not.

Core Claim

A protein language model's per-residue embeddings already carry enough information to discriminate alternative folding states — they implicitly encode structural contacts without any 3D supervision, as shown by Rao et al. (Transformer protein language models are unsupervised structure learners). The single-state bottleneck is therefore not an information problem but a representational one: the folding head collapses all of that latent variation into one output. We claim that replacing the head's monolithic conditioning with a prototype bank — residues softly assigned to one of $K$ learned structural prototypes, each prototype steering a shared structure decoder — lets a single forward pass express multi-modal conformational output, including fold switching, at alignment-free speed. Ablations in vision show multi-prototype banks beat single-prototype ones by 10–20% on multi-modal 3D tasks (Few-Shot 3D Point Cloud Semantic Segmentation); we expect the same mechanism to transfer, since the failure mode (one vector forced to average over deeply distinct modes) is identical.

Research Program

The system, ProtoFold, modifies a frozen alignment-free predictor (ESMFold, following the adapter pattern of ProSAM, which showed that a ~12M-parameter side module — about 1.9% of the base model — can recover most of the gap to AlphaFold2 while adding only ~1 second per protein; ProSAM: Modular and Energy-Guided Fine-Tuning of Protein Language Models for Structure Prediction). Between the frozen ESM-2 trunk and the folding head we insert a global bank of $K$ structural prototypes $\{c_1, \dots, c_K\}$, trained jointly with a lightweight gating module. Each residue embedding $z_i$ receives a soft assignment over the bank, using the same smoothed-softmax form that Qian et al. showed is the natural multi-center generalization of ordinary classification (SoftTriple Loss: Deep Metric Learning Without Triplet Sampling):

$$ p_{ik} = \frac{\exp(\langle z_i, c_k \rangle / \tau)}{\sum_{j=1}^{K} \exp(\langle z_i, c_j \rangle / \tau)}, \qquad \mathcal{L}_{\text{div}} = \lambda \, \|C\|_{2,1}, $$

where $\tau$ is a temperature and the $L_{2,1}$ sparsity penalty on the prototype matrix $C$ prunes redundant prototypes — the device SoftTriple uses to prevent all $K$ vectors from collapsing onto the same mode, which the multi-prototype literature identifies as the central failure risk. The prototype-weighted conditioning is then fed to $K$ decodes of the (shared, frozen) folding head, producing $K$ candidate structures in one pass, each ranked by an assignment-entropy confidence score in the spirit of EigenFold's ELBO ranking, but without any sampling.

Three design questions structure the research program. First, cardinality: a fixed $K$ is the wrong prior, since fold-switching proteins need at least two meaningful modes while most single-fold proteins need one; we will adapt $K$ per protein by pruning prototypes whose global assignment mass falls below a threshold, borrowing the adaptive-allocation logic of ASGNet, which discovered that letting the data choose the prototype count beats fixing it a priori (Adaptive Prototype Learning and Allocation for Few-Shot Segmentation). Second, supervision: single-structure training data (TEDBench's 462,175 high-confidence structures, from Protein Fold Classification at Scale: Benchmarking and Pretraining) cannot teach a bank to cover multiple basins, so prototype diversity is additionally trained against the ATLAS molecular-dynamics trajectories, the same 1,390-target ensemble resource used by ESMFLOW (AlphaFold Meets Flow Matching for Generating Protein Ensembles), by requiring the decoded set to cover ATLAS conformations rather than match one. Third, point versus distributional prototypes: Zhu et al. argue that deterministic point proxies create gradient conflicts when a class has high internal variance and should be replaced by continuous distributions (Absorbing Gradient Conflicts: Modeling Semantic Variance via Kent Distributions for Cross-Modal Hashing); because a conformational basin is genuinely variance-rich, we treat this as an open ablation — point prototypes versus small Gaussian-mixture prototypes, in the spirit of Ding et al.'s Gaussian-mixture scene-graph prototypes (Learning Gaussian Mixture-distributed Prototypes for 3D Scene Graph Generation from RGB-D Sequences) — rather than assuming point prototypes suffice.

A secondary analysis examines interpretability: because prototypes are global and shared, they should spontaneously organize around CATH fold topologies, giving a multi-prototype readout for fold recognition that contrasts with the single-vector global pooling used by MiAE on TEDBench (Protein Fold Classification at Scale: Benchmarking and Pretraining).

Baselines

The do-nothing reference point is frozen ESMFold predicting its single structure. On the post-training-cutoff temporal PDB benchmark of Hyskova et al. (1,337 structures deposited July 2022–July 2024, filtered at ≤70% sequence identity; Balancing speed and precision in protein folding: a comparison of AlphaFold2, ESMFold, and OmegaFold), ESMFold reports a median TM-score of 0.95, median GDT-TS of 90%, and median RMSD of 1.74 Å, against 0.96 / 94% / 1.30 Å for MSA-dependent AlphaFold2. Any multi-prototype modification must not regress these single-structure numbers — a decoded top-ranked structure within 0.01 median TM-score of frozen ESMFold counts as parity. The file's category-specific breakdowns for monomers, complexes, and de novo proteins are reported only as narrative, so we will re-run ESMFold ourselves on those splits rather than cite figures that do not exist.

The strongest competitor for ensemble behavior is Energy-Based Alignment, which fine-tunes a diffusion ensemble generator with Boltzmann-weighted trajectories and reports 0.90 per-target RMSF correlation on ATLAS, against roughly 0.71 for prior generative models (Aligning Protein Conformation Ensemble Generation with Physical Feedback); AlphaFLOW, reporting median RMSF correlation 0.85 with a 10× speedup over replicate MD, is the second competitor (AlphaFold Meets Flow Matching for Generating Protein Ensembles). To be fair we will run them at the same total budget of decoded structures per protein (equal wall-clock for us means $K$ parallel single-pass decodes, versus their iterative sampling) and tune all methods with the same validation targets drawn from ATLAS. Beating them means matching or exceeding RMSF correlation at a substantially lower per-sample cost, and, more decisively, higher coverage of discrete alternate conformations (fraction of MD cluster centers within a 2.5 Å RMSD of some decoded candidate), which is where discrete prototypes should have an edge and continuous fluctuation models should not.

For metamorphic-protein classification, the strongest published system is the SMICE-plus-statistics pipeline of Chen et al., which perturbs AlphaFold2's MSAs, clusters the resulting structures, and classifies metamorphism at a mean AUC of 0.869 with 79.6% sensitivity (Classifying Metamorphic versus Single-Fold Proteins with Statistical Learning and AlphaFold2). It will be run on the metamorphic benchmark's 80 verified fold-switching proteins and 306 single-fold proteins wherever an MSA can be built; our alignment-free variant uses prototype-assignment statistics (spread and entropy of the assignment over the bank) as the classification signal. The strongest alignment-free single-structure competitor is ProSAM itself (ProSAM: Modular and Energy-Guided Fine-Tuning of Protein Language Models for Structure Prediction), which reports +4.1 TM-score points over ESMFold on CASP14 and recovery of 74% of the ESMFold–AlphaFold2 gap; we will match its adapter parameter count (~12M) and tuning budget exactly. Finally, the key ablation is the same architecture with $K = 1$: one prototype collapses the design back into ProSAM-style single-prompt conditioning, so any gain over $K{=}1$ isolates the multi-prototype ingredient itself. Random (untrained) prototypes and fixed versus adaptive $K$ complete the ablation set.

Why not just…?

Why not just subsample MSAs, as SMICE already does? This is the closest existing solution to the multi-state problem and must be taken seriously. But MSA subsampling is exactly what alignment-free prediction exists to avoid: it fails outright on orphan proteins, de novo designs, and fast-evolving sequences, as both Hyskova et al. and the AIDO.RAGFold authors document (Balancing speed and precision in protein folding; Retrieval Augmented Protein Language Models for Protein Structure Prediction), and Jing et al. call it an inconsistent hack that cannot be trained against true dynamics distributions (AlphaFold Meets Flow Matching for Generating Protein Ensembles). The decisive experiment is the same benchmark SMICE uses: if ProtoFold reaches comparable AUC (0.869 is the target) without any alignment, and SMICE cannot run at all on the orphan/de novo subset where we can, the alignment-free claim is settled. On targets with deep MSAs, SMICE may well stay ahead, and we will report that honestly.

Why not just use a diffusion or flow-matching ensemble model? EigenFold and ESMFLOW already generate multiple structures from a single sequence, so the question is whether discrete prototypes add anything. Three documented shortfalls say yes for our target regime: they need 100–300 sequential steps (EigenFold: Generative Protein Structure Prediction with Diffusion Models), forfeiting the seconds-per-protein advantage; EigenFold's authors concede the sampled diversity is predominantly epistemic model uncertainty inherited from single-state conditioning, not real conformational multimodality; and continuous flows struggle to cross deep energy barriers between distinct folds without physical fine-tuning (Aligning Protein Conformation Ensemble Generation with Physical Feedback). The honest threat is the converse: for smoothly fluctuating proteins, continuous models plausibly model the ensemble better than a few discrete prototypes, and EBA's 0.90 RMSF correlation is a high bar. Our experiments directly test this by splitting ATLAS targets into near-unimodal and multi-basin subsets and reporting per-split performance; if prototypes lose on RMSF in the unimodal split, that is the expected boundary of the method, not a failure of the proposal.

Why not just use a single centroid or single prompt, as everyone else does? MiAE classifies folds with a single pooled vector and ProSAM conditions folding with a single fold prompt, and both work. But the multi-prototype literature quantifies the cost of this collapse on exactly analogous geometry: single-center SoftMax reaches 57.8% Recall@1 on fine-grained retrieval versus 65.4% for SoftTriple's multiple centers (SoftTriple Loss), and multi-prototype representations beat single-prototype ones by 10–20% on 3D point-cloud tasks (Few-Shot 3D Point Cloud Semantic Segmentation). The $K{=}1$ ablation makes the same comparison inside our own architecture; if it matches the multi-prototype version on every metric, the idea is wrong and we will say so.

Why not just use external secondary-structure predictors to guide regions, as ProSAM does? External tools like NetSurfP can identify loops and cores without new machinery, and ProSAM's authors themselves note the cost: upstream prediction errors cascade into the folding trunk, and the fold prompt remains a uniform task prompt rather than a dictionary of topologies (ProSAM: Modular and Energy-Guided Fine-Tuning of Protein Language Models for Structure Prediction). Our prototypes are learned end-to-end from folding signal alone — no external annotations — and, unlike 1D secondary-structure labels, they index whole 3D basins, which is the level at which fold switching happens. An ablation replacing learned prototypes with NetSurfP-conditioned prompts separates the two explanations.

Proposed Experiments

The first experiment tests single-structure fidelity: on Hyskova et al.'s temporal PDB benchmark (Balancing speed and precision in protein folding), the top-ranked decoded structure must stay within 0.01 median TM-score of frozen ESMFold's 0.95, with runtime within ~1 second of it (ProSAM reports 4.9s versus 3.8s per protein for the base system, a budget we match). The second experiment tests ensemble coverage on ATLAS (AlphaFold Meets Flow Matching for Generating Protein Ensembles): per-target RMSF correlation against AlphaFLOW's reported 0.85 and EBA's 0.90, plus the discrete-basin coverage metric, reported on unimodal and multi-basin splits separately. The third experiment is the metamorphic benchmark of Chen et al. — 80 metamorphic and 306 single-fold proteins — scored as alignment-free classification and compared against their MSA-dependent AUC of 0.869 and 79.6% sensitivity, with an explicit orphan/de novo subset where the MSA baseline cannot run; the file reports no result for any alignment-free method on this benchmark, so any working number here is new evidence. The fourth experiment attaches a multi-prototype readout for CATH fold classification on TEDBench (Protein Fold Classification at Scale), where the numbers to beat are MiAE-B at 74.56 macro-F1 and SaProt-650M at 73.48. The decisive checks are the ablations: $K{=}1$, random prototypes, fixed versus adaptive cardinality, and point versus Gaussian-mixture prototypes. If $K{=}1$ matches $K{>}1$ across experiments, the multi-prototype hypothesis is refuted in this setting; if adaptive prototypes beat fixed ones and trained beat random ones while single-structure accuracy holds, the claim stands.