Protein language models — transformers trained on hundreds of millions of amino acid sequences — have learned to read the evolutionary "grammar" of proteins, but they assign functions the way a text classifier assigns topic IDs: each Enzyme Commission (EC) number or Gene Ontology term is just an integer in a fixed output head. Meanwhile, in natural language processing, a line of work on semantic label embedding has shown that when you instead encode the textual definition of each class and match inputs against those definitions in a shared metric space, models can classify classes they never saw a training example for — label-description training alone lifted zero-shot text classification accuracy by 17.5% to 18.9% absolute in The Benefits of Label-Description Training for Zero-Shot Text Classification. This proposal asks whether that transfer is possible in biology, where it would matter most: metagenomic sequencing routinely yields enzymes whose exact activity has zero previously sequenced representatives, and every current method needs at least one.
The closest existing system, EnzPlacer How Not to be Seen: Predicting Unseen Enzyme Functions using Contrastive Learning, handles unseen EC leaf classes by aligning protein embeddings to sequence centroids — the average embedding of known proteins sharing each EC prefix. It reaches 43.5% placement accuracy at the EC2 level on its Unseen-EC4 benchmark (versus 38.5% for CLEAN). But its anchors are made of sequences: when an entire functional branch has no known members, no centroid exists and the method is undefined. My core claim is that the textual definitions themselves — the Enzyme Commission's accepted names, reaction formulas, and catalytic comments — are a sufficient anchor. A protein embedding aligned into a metric space with text embeddings of EC definitions can place a protein into an enzymatic class that has never had a single sequenced representative, because the description of the reaction ("hydrolysis of a phosphoanhydride bond," for example) is available even when the sequences are not.
Core Claim
A dual-encoder with a frozen protein language model on one side and a frozen biomedical text encoder on the other, trained contrastively to pull proteins toward the text descriptions of their EC paths, will (a) match or beat sequence-centroid placement on unseen EC4 classes, and (b) remain defined — and empirically useful — on truly exemplar-free classes where EnzPlacer and all homology-based methods cannot produce a prediction at all. The falsifiable version: on the Unseen-EC4 benchmark, text-anchored placement should exceed the 43.5% EC2 and 35.6% EC3 accuracy EnzPlacer reports, and on a newly constructed split holding out entire EC3 branches, it should achieve placement accuracy strictly above the BLASTp and prompted-LLM reference points.
Research Program
Three things get built. First, a label catalogue: for every node in the EC hierarchy, I will assemble a structured description from the ExPASy ENZYME nomenclature — accepted name, reaction formula, cofactor and comment fields — and encode it with PubMedBERT, the biomedical text encoder that OntoProtein: Protein Pretraining With Gene Ontology Embedding chose for GO definitions. Each EC path (EC1 through EC4) thus has a text anchor $d_c$ that exists independent of any training sequence. Second, the protein tower: a frozen ESM-1b encoder (matching EnzPlacer's backbone for comparability) with a trainable MLP adapter, mean-pooled to a sequence embedding $z$. Third, the alignment objective. For a protein with EC label $y$, the loss is a temperature-scaled contrastive term over hard negatives $N(y)$ drawn from siblings at varying depths in the EC tree:
$$\mathcal{L} = -\log \frac{\exp(\text{sim}(z, d_y)/\tau)}{\exp(\text{sim}(z, d_y)/\tau) + \sum_{c \in N(y)} \exp(\text{sim}(z, d_c)/\tau)}$$
where $\text{sim}$ is cosine similarity and $\tau$ a learned temperature. Two design elements address documented failure modes. Sibling negatives at the deepest shared level counter the granularity-collapse problem — EC4 descriptions within one EC3 often differ only by substrate name, so text embeddings will blur unless explicitly forced apart (the same ambiguity noted as an open problem in the label-embedding literature). And a hierarchy-consistency penalty, adapted from the child-above-parent penalty in TALE: Transformer-based protein function Annotation with joint sequence–Label Embedding, is charged whenever a query scores a child description above its own parent's description, since a protein that hydrolyzes phosphoanhydrides must also score the broader hydrolase class.
A central design question, tested rather than assumed, is how to combine text anchors with sequence information when exemplars do exist. The plan is a hybrid inference rule: where a branch has sequence centroids available (the EnzPlacer regime), the score is a weighted blend of text-anchor similarity and centroid similarity; where it does not, the text anchor alone carries the prediction. The blend weight is fit on validation data as a function of the number of exemplars, which turns "text vs. sequences" into a measured curve rather than an ideological choice. I expect text anchors to dominate in the exemplar-poor and low-identity regimes and centroids to dominate for abundant, close-homology classes — consistent with the pattern in TALE, where homology tools still win on Molecular Function tasks driven by short conserved motifs.
One honest risk deserves pre-emptive treatment: sensitivity of text anchors to phrasing, the brittleness documented for verbalizers in The Benefits of Label-Description Training for Zero-Shot Text Classification. Following that paper's remedy, I will fine-tune the text encoder on a corpus of EC nomenclature — definitions, synonyms, reaction variants, and paraphrases — before contrastive alignment, so the anchor encoder has seen the register of enzymological language rather than only general biomedical text.
Proposed Experiments
The primary evaluation is the Unseen-EC4 benchmark from How Not to be Seen: roughly 183,000 single-function proteins from ExPASy ENZYME mapped to UniProtKB, with abundant EC4 groups in training and rare EC4 groups held out for test, partitioned by BLASTp identity into <50%, <30%, and <10% bins. Metric: placement accuracy at EC2 and EC3. The numbers to beat are EnzPlacer's reported 43.5% (EC2), 35.6% (EC3), and 30.3% (EC2 at <10% identity). Second, I will construct a "Zero-EC" split by removing entire EC3 branches from training — the regime where EnzPlacer is structurally undefined — and report accuracy against BLASTp transfer and a prompted large language model as reference points; this split is where the proposal's unique capability lives. Third, to show no regression on conventional annotation, I will evaluate on New-392 and Price-149, where Predicting Enzyme Functions Using Contrastive Learning with Hierarchical Enzyme Structure Information reports F1 scores of 0.5356 and 0.5121 respectively for EnzHier. Fourth, robustness checks: paraphrase the held-out EC descriptions and measure accuracy degradation (testing the phrasing-brittleness concern directly), and report the hierarchy-violation rate (fraction of queries scoring a child above its parent) with and without the consistency penalty.
Baselines
The naive reference point is BLASTp nearest-neighbor transfer: annotate a query with the EC number of its best database hit. It is the default practice in metagenomics and the floor any learned method must clear; on the Unseen-EC4 splits at <10% identity it collapses, which is precisely the regime this proposal targets, so beating it there is expected and mainly quantifies the opportunity. The strongest existing competitor is EnzPlacer How Not to be Seen, which reports 43.5% EC2 placement accuracy on Unseen-EC4 versus 38.5% for CLEAN. To make the comparison fair I will use the same frozen ESM-1b backbone, the same training pool and split construction, and a matched adapter parameter count and tuning budget (grid over temperature, negative-sampling depth, and learning rate on the same validation fold), so the only systematic difference is the anchor type: text embeddings of EC definitions versus sequence centroids. Beating it means exceeding 43.5% at EC2 and 35.6% at EC3 on the identical split, and — separately — producing any non-trivial accuracy at all on Zero-EC branches where it cannot run. For the seen-class experiments, EnzHier Predicting Enzyme Functions Using Contrastive Learning with Hierarchical Enzyme Structure Information is the competitor to match on New-392 (F1 0.5356) and Price-149 (F1 0.5121) under the same data. Finally, two ablations isolate the proposal's key ingredient: replacing text anchors with randomly initialized label vectors (which should collapse zero-shot performance to chance and confirm the semantics live in the descriptions, not the architecture), and the hybrid-versus-text-only-versus-centroid-only inference comparison, which measures exactly how much each anchor type contributes at each exemplar-count level.
Why not just…?
Why not just use BLASTp or DIAMOND? Because the target regime is where alignment fails. Three independent results in the file document the collapse: in TALE, homology methods lose predictive power below 30% sequence identity; in Prot2Text-V2: Protein Function Prediction with Multimodal Contrastive Alignment, BLAST-based transfer bottoms out in the bitscore 0–30 regime; and in How Not to be Seen, BLASTp collapses below 10% identity where learned representations still reach 30.3% EC2 accuracy. To be fair, alignment tools win some regimes — the alignment baseline hit 0.80 accuracy on secondary structure in Evaluating Protein Transfer Learning with TAPE — so BLASTp stays in every experiment as the reference, and the claim is scoped to low-homology and exemplar-free splits, not to annotation in general.
Why not just train a linear head on a frozen ESM-2? A classification head is arithmetically incapable of the target task: it has one weight vector per training class, so an EC number absent from training can never be predicted, no matter how good the representation. The tail-label problem compounds this even within the training vocabulary — TALE documents standard heads failing on rare GO terms. The linear head will still be run on New-392 and Price-149 to confirm it matches on seen classes while producing nothing on Unseen-EC4 and Zero-EC; that contrast is the cleanest demonstration of what label semantics buy.
Why not just do flat supervised contrastive learning (CLEAN) with k-nearest-neighbor search? CLEAN treats EC numbers as arbitrary cluster IDs and predicts via nearest exemplars; on Unseen-EC4 it achieves 38.5% EC2 accuracy versus EnzPlacer's 43.5%, and it shares the exemplar dependence — with zero training instances of a class, no neighbor can vote for it. The hierarchy-aware variant EnzHier improves margins via tree distance but still grounds labels in nothing but their position in the tree. The centroid-only ablation in this proposal is essentially a controlled replication of that family, so the experiments themselves will show how much the reaction-chemistry text adds beyond hierarchy structure alone.
Why not just generate a functional caption with a protein-to-text model and parse out the EC number? This is a real threat worth naming. Prot2Text-V2 reports a BLEU-4 of 43.34 on Swiss-Prot captioning, and Automated Protein Function Description for Novel Class Discovery shows generation models can rank descriptions for unseen functions 21% above chance. But both leave open exactly what this proposal fixes: Barot et al. themselves note their outputs produce terms outside formal ontologies and require expert manual curation, and surface metrics like BLEU do not certify that the generated EC number is the correct one. The dual-encoder is not obviously better at captioning — it is better at falsifiable placement: its output is a ranked list over the actual EC catalogue, scored against ground truth. I will run caption-and-parse as a baseline on the Zero-EC split using the reported Prot2Text-V2 recipe; if it matches text-anchored placement accuracy there, the simpler generative route wins and I will say so.
Decisive Checks
The proposal succeeds if three things hold: text-anchored placement exceeds EnzPlacer's 43.5% EC2 / 35.6% EC3 on Unseen-EC4 under matched conditions; accuracy on the Zero-EC split is materially above the BLASTp and caption-and-parse reference points, establishing a capability that currently does not exist; and the paraphrase test shows degradation under, say, five description rewrites of no more than a few points, confirming the anchors encode chemistry rather than phrasing. It fails informatively if sibling-level text anchors cannot be separated — the granularity-collapse scenario — in which case the hierarchy-violation and per-depth error analyses will show whether the bottleneck is the label encoder or the alignment objective, and whether substrate-aware negatives or reaction-formula anchors close it. Either outcome settles, with numbers, whether enzymatic function can be predicted from a description of the reaction alone — a question the compiled literature raises at every turn but never tests.
References
- The Benefits of Label-Description Training for Zero-Shot Text Classification
- How Not to be Seen: Predicting Unseen Enzyme Functions using Contrastive Learning
- OntoProtein: Protein Pretraining With Gene Ontology Embedding
- TALE: Transformer-based protein function Annotation with joint sequence-Label Embedding
- Predicting Enzyme Functions Using Contrastive Learning with Hierarchical Enzyme Structure Information
- Prot2Text-V2: Protein Function Prediction with Multimodal Contrastive Alignment
- Evaluating Protein Transfer Learning with TAPE
- Automated Protein Function Description for Novel Class Discovery