Fewer than one percent of known proteins have experimentally confirmed functions. Machine-learning predictors that read amino-acid sequences are trained almost entirely on that small curated set, so they do worst on the specific, rare functions biologists care about most. A second, much larger source of evidence goes unused by these predictors: the biomedical literature mentions hundreds of thousands of proteins whose functions no curator has ever entered into a database. Text-mining systems can turn those mentions into labels automatically. This approach is called distant supervision, meaning training labels produced by aligning raw text with an existing database instead of by hand. The trouble is that such labels are noisy. A protein named next to "kinase" is not necessarily a kinase. Standard noise filters also tend to throw out exactly the rare labels that would help most, because rare labels look uncertain to any model.
We propose a simple division of labor. Text supplies the fine-grained function, and a sequence model checks only the coarse category that function belongs to. Suppose a literature miner says protein P has a rare, specific Gene Ontology term g. We keep that label only if a sequence model, trained on curated data alone, confidently assigns P to the nearest broader category of g that has plenty of curated training examples. Sequence models are reliable at coarse placement even for distant proteins, and text is reliable at naming specific activities. Text-mining errors (trigger words, co-mentions) and sequence-model errors (homology confusion) come from different sources, so their agreement at the coarse level should screen out noise without penalizing rarity. The filtered labels then train a sequence-only predictor. That predictor can annotate the "dark matter" proteins that have no literature at all.
Core Claim
Gating literature-derived labels on agreement with a sequence model at a well-populated ancestor term produces a cleaner and less frequency-biased distant training set than gating on model confidence for the term itself. A sequence-only predictor trained on curated labels plus these gated labels will improve rare-term accuracy on held-out, literature-free proteins. The comparison is against the same predictor trained on curated labels only, on ungated distant labels, or on labels cleaned by generic noise filters.
To make the gate concrete, let $t(p,g)$ be the literature miner's score that protein $p$ has term $g$. Let $a_N(g)$ be the deepest ancestor of $g$ in the Gene Ontology graph with at least $N$ curated training proteins. Let $s(p,a)$ be the sequence model's calibrated probability for term $a$, and let $\tau_a$ be the threshold that gives a fixed precision for $a$ on curated validation data. A distant label is kept with weight
$$w(p,g) = t(p,g)\cdot \mathbf{1}\left[s(p, a_N(g)) \ge \tau_{a_N(g)}\right].$$
The threshold $N$ controls the trade-off. When $N$ is too small, the check reverts to the unreliable same-term confidence. When $N$ is too large, the check becomes the near-useless root node.
Why this is the gap, and what prior work does
Literature-side distant supervision for protein function already works well enough to be credible, but it never reaches sequences without papers. Zhu et al. mined GeneRIF for noisy negatives and projected curated GO terms through retrieved articles, with hierarchical F1 of 0.301 at best on BioCreative IV. GOAnnotator replaced curator-chosen papers with automatic MEDLINE retrieval and matched curated-literature systems on SP2024 and TR2024. Elangovan et al. extracted 1.6 million modification interactions from PubMed. These papers motivate the proposal: they show text can yield many function labels for proteins curators missed. They also constrain it. Elangovan et al. report that precision fell from 100% on a controlled test set to 33.7% across all of PubMed, and they attribute the remaining errors to trigger-word reliance. Their ensemble-variance filter and multi-paper requirement are therefore our main denoising baselines. None of these systems feeds its output into a sequence model. We do exactly that.
Sequence-side predictors supply the method we train. NetGO 3.0 showed that a logistic-regression head on ESM-1b embeddings (LR-ESM, where ESM-1b is a protein language model pretrained on more than 250 million sequences) drives most of its gains. TALE showed that sequence-only models hold up below 30% identity. Both learn only from curated labels, and the long tail is their weak point: the CAFA1 assessment found roughly 38% of experimental data in the single term "protein binding". Our premise that coarse placement is reliable comes from EnzPlacer, which places enzymes with unseen fourth-level EC numbers into second-level classes at 43.5% accuracy, and 30.3% even below 10% identity, where BLASTp collapses. Our premise that hierarchy can smooth confidence comes from Tail-GNNs, which passes messages over the ontology graph at the output layer. We use the hierarchy to decide which labels to trust, not to smooth predictions.
On the noise-filtering side, DIPS discards samples whose training-time confidence is low or unstable. It reports +20% accuracy over plain pseudo-labeling at 30% noise, but it was tested only on tabular and image data, and it degrades beyond about 45% noise. The 33.7% PubMed precision suggests distant GO labels may sit well past that point. A positive rare-term label also looks "low confidence, high variance" by construction. Our central empirical bet is that DIPS-style filtering disproportionately deletes correct tail labels, and that ancestor gating does not. This is a measurable disagreement with a strong method, not an assumption. SLEEC is the nearest example of biology-specific pseudo-labeling. It labels active-site residues from alignment conservation and pools embeddings over them, but its pseudo-labels are residue-level evolutionary signals, not literature-derived function labels. ProtNote attacks rare and unseen terms differently, by matching sequences to term descriptions in a shared space (zero-shot GO mAP macro 0.2139). It is a competitor for the tail, not a source of labels.
Research Program
The pipeline has four stages. First, a literature miner in the style of GOAnnotator is run only over MEDLINE abstracts published before the training cutoff. It covers proteins that lack experimental GO annotations in Swiss-Prot at that date, mostly TrEMBL entries with at least one textual mention, and outputs scored (protein, term) pairs. The date restriction is essential. Without it, papers that later justified test-set annotations would leak into training. Second, a "gatekeeper" sequence model (frozen ESM embeddings with a per-term logistic head, as in LR-ESM) is trained on curated pre-cutoff labels and calibrated per term on a validation split. Third, every distant label is gated by its ancestor $a_N(g)$, sweeping $N$ over a small grid chosen on validation data. Fourth, a "student" with the same architecture is trained on curated labels plus weighted gated labels. It is then evaluated on held-out proteins that are homology-filtered against all training proteins and, for the main analysis, have no literature mentions before the cutoff.
Label quality can be measured without experts for most of the data. Some proteins received distant labels before the cutoff and experimental curation afterward. For these, later curation acts as a free oracle for per-label precision, broken down by term frequency and by gate decision. Experts are used only to audit a small stratified sample, about 500 labels, from proteins never curated afterward. This audit checks whether the temporal oracle is biased toward well-studied proteins. The scarce resource, expert time, is spent on calibration rather than labeling.
Baselines
The naive reference point is the student architecture trained on curated pre-cutoff labels alone, alongside DIAMOND homology transfer. Together these answer whether any distant data helps at all, and they show how much of the tail is reachable by alignment. They share the encoder, label space, epochs, and hyperparameter search budget of every other learned condition.
The strongest published sequence predictor in the file is NetGO 3.0. On its time-split benchmark (trained on pre-2020 annotations, tested on 2021), it reports biological-process Fmax of 0.378, and Fmax of 0.369 on evolutionarily distant "difficult" proteins (NetGO 3.0). We adopt that split and its cutoffs so this number is directly comparable. We also retrain LR-ESM ourselves as the in-house equivalent. Beating it means a higher Fmax overall, and more importantly a higher macro-averaged precision-recall area on terms with fewer than 50 curated proteins, with confidence intervals from bootstrap over proteins.
The strongest denoising competitors operate on the identical distant label pool. These are ungated distant labels, the ensemble-variance and multi-paper filters of Elangovan et al., and DIPS applied per term (DIPS). Each filter is tuned to keep the same total number of labels as our gate, so that the comparison is about which labels are kept, not how many. The method only beats them if it wins on tail terms at matched label count. A win confined to frequent terms does not count.
For rare and unseen terms, ProtNote is the direct competitor. It reports supervised GO mAP macro of 0.6018 and zero-shot mAP macro of 0.2139 on terms introduced after July 2019 (ProtNote). We run it on our split with its released encoders and also test a combination: ProtNote trained with our gated labels added. GOAnnotator's direct literature predictions serve as a ceiling reference on the subset of test proteins that do have literature. No numeric F1 for it appears in the file, so no reported result was found on Lacuna for GOAnnotator on SP2024/TR2024 to quote.
Three ablations isolate the key ingredient. In the first, the gate checks $s(p,g)$ for the term itself rather than its ancestor. In the second, the gate checks a randomly chosen ancestor at matched depth. In the third, the sequence check is kept but the text score is replaced with uniform weight. If same-term gating performs as well as ancestor gating, the hierarchical idea is wrong.
Why not just…?
Why not just train on more curated data and homology transfer? For proteins with close curated relatives this may be all that is needed, which is why DIAMOND is a baseline. The evidence against it as a full answer is that alignment-based transfer collapses below 30% identity (TALE) and below 10% on unseen enzyme classes (EnzPlacer). Curated data also cannot grow faster than curation. We report results stratified by identity bin. If gains vanish everywhere except high-identity bins, distant labels are only re-teaching homology, and we will say so.
Why not just filter the distant labels with ensemble variance or DIPS? This is the real threat. It is cheaper and already validated, with 100% precision at 19% retention on a controlled set (Elangovan et al.) and +20% under 30% noise (DIPS). Our argument that these filters penalize rarity is a hypothesis, and the temporal-oracle analysis tests it directly by measuring, for each filter, the precision and recall of kept labels as a function of term frequency. If DIPS keeps tail labels at the same rate and precision as the ancestor gate, the proposal reduces to "apply DIPS to protein data." We would then report that result rather than the gate.
Why not just use the literature miner's predictions as annotations, or use ProtNote's zero-shot scores for the tail? Literature predictions exist only for proteins with papers. Our main test set is literature-free by design, and that is where text alone, as in GOAnnotator, says nothing. ProtNote does reach literature-free proteins, but it relies on curated term descriptions, and its zero-shot mAP macro of 0.2139 leaves room. The decisive check is whether gated labels improve ProtNote itself. If they do not, text-description matching already captures what literature labels would add.
Why not just train on the existing automatic database annotations? UniProt already carries large volumes of electronically inferred GO terms. A reviewer will reasonably ask whether these are a free, cleaner weak-label source. Most such annotations are themselves propagated from sequence similarity or family signatures. They are therefore correlated with the gatekeeper's own errors and add little independent evidence, whereas literature labels come from a different channel. We include them as an extra weak-label condition under the same gate, so this argument is tested rather than asserted.
Proposed Experiments
The primary experiment uses the NetGO 3.0 time split, restricted to test proteins that have no pre-cutoff literature mentions and are below 30% identity to any training protein. Metrics are Fmax and Smin per ontology, plus macro precision-recall area on frequency buckets: fewer than 10, 10 to 50, and more than 50 curated proteins. The decisive result is a gain over both the curated-only student and the best matched-count denoising filter in the rarest bucket, with bootstrap intervals that exclude zero. A secondary replication on the CAFA5 temporal targets and TALE's identity-stratified Swiss-Prot split checks that the effect is not specific to one cutoff.
The mechanism check is the temporal-oracle analysis. For each filter, we plot the precision of kept labels and the fraction of truly correct labels kept, both against term frequency. The hypothesis predicts that the ancestor gate's retention of correct labels stays roughly flat with frequency while DIPS's falls in the tail. The expert audit of about 500 labels confirms that the precision ordering holds on proteins never curated later. Finally, sweeping $N$ should trace an inverted U in tail accuracy: too low collapses to same-term gating, and too high admits noise. A flat curve would mean the hierarchy is doing no work and the idea should be abandoned.
References
- Integrating information retrieval with distant supervision for Gene Ontology annotation
- GOAnnotator: accurate protein function annotation using automatically retrieved literature
- Large-scale protein-protein post-translational modification extraction with distant supervision and confidence calibrated BioBERT
- NetGO 3.0: Protein Language Model Improves Large-scale Functional Annotations
- TALE: Transformer-based protein function Annotation with joint sequence-Label Embedding
- A large-scale evaluation of computational protein function prediction
- How Not to be Seen: Predicting Unseen Enzyme Functions using Contrastive Learning
- Hierachial Protein Function Prediction with Tails-GNNs
- You can't handle the (dirty) truth: Data-centric insights improve pseudo-labeling
- Enzyme Classification via Semi-Supervised Functional Residue Learning
- ProtNote: a multimodal method for protein-function annotation