Checking Domain Completeness Against Text: Clause-Level Verification for Gene Family Discovery

When biologists catalog a gene family in a newly sequenced genome, they first cast a wide net for candidate proteins. The tools are profile models (HMMER) and pairwise alignment (BLASTP). They then decide by hand which candidates are real, intact family members and which are fragments, pseudogenes, or look-alikes, mostly by running each candidate through a domain database such as NCBI CD-Search. The second step is where expertise and time go. It also breaks down exactly where new neural search tools are most useful. A distant relative found by a protein language model (a neural network trained on millions of protein sequences) usually scores below the domain database's threshold, so nothing can confirm that it really carries the family's defining features.

We propose to make that confirmation step learned and readable. A family's textual definition is split into short clauses, for example "zinc-coordinating cysteines in a B-box motif," "membrane-spanning helices," or "an ACT-like dimer interface." Each clause is matched against the individual amino-acid positions of a candidate protein. A candidate is accepted only if every required clause finds strong, well-localized support somewhere in the sequence. The output is both an accept/reject decision and a map of which residues satisfy which part of the definition. Because clauses such as "transmembrane helix" or "zinc binding" recur across thousands of families, the matcher can be trained once and applied to families it never saw.

Core Claim

Residue-level "late interaction" can verify domain completeness for remote family members in families held out from training, with fewer false accepts of truncated fragments than either pooled sequence–text models or sequence-to-sequence retrieval followed by standard domain checks. Late interaction means scoring a match by letting each token on one side find its best-matching token on the other, rather than comparing two whole-sequence summaries. Here the scoring runs between the text tokens of each definition clause and the protein's residue embeddings. The claim is deliberately narrow. Text is not proposed as a better way to find candidates, since sequence exemplars probably remain better for recall. Text is proposed as a better way to verify and localize them when no curated profile covers them.

The motivating tension is in the literature. ProtST aligns protein language models with biomedical text through contrastive training on 553,052 Swiss-Prot sequence–text pairs, and shows zero-shot function classification from text prompts. It compresses each protein into one mean-pooled vector. ProtoCol shows that this pooling discards the local signals needed in the twilight zone, meaning sequence identity below roughly 25%. Its ColBERT-style per-residue MaxSim matching with a 35M-parameter ESM-2 beats a pooled 650M ESM-2 by nearly seven points on SCOPe superfamily retrieval. ProtoCol works only from sequence to sequence and never sees text. TALE embeds Gene Ontology terms jointly with sequences and reports that this global embedding fails on molecular function, which depends on short local motifs. Together these papers suggest text supervision helps transfer and residue-level matching helps local discrimination. No work in the file combines the two, and none aims the combination at the filtering step of gene family pipelines.

Research Program

The model has two encoders. ESM-2 produces one embedding per residue, and a biomedical text encoder produces one embedding per token of each definition clause. Following ProtoCol, both sets are projected into a shared space. For a candidate protein $x$ with residue embeddings $e_1,\dots,e_L$, and a clause $a$ with token embeddings $u_1,\dots,u_{T_a}$, the clause score is

$$s_a(x) = \frac{1}{T_a}\sum_{t=1}^{T_a} \max_{i \le L} \cos(e_i, u_t).$$

The family-level decision is

$$\text{accept}(x) \iff \min_{a \in A_F} s_a(x) > \tau_F,$$

where $A_F$ is the set of required clauses for family $F$ and $\tau_F$ is a threshold calibrated on held-out families. The minimum is the key design choice. A fragment that keeps the fold but loses the active site should fail one clause even if its average similarity is high. A pooled model cannot express this failure. Each residue that wins a $\max$ also gives a residue-level localization for free. ProtoCol truncates inputs at 256 residues; we instead score overlapping windows and take the maximum over windows. This matters because family members such as the 219–1,140-residue OsKEA transporters in the rice KEA study are often longer than 256 residues.

The training data come from Swiss-Prot entries, the same source as ProtDescribe. Their family and function descriptions are split into clauses. Wherever position-level annotations exist (active sites, binding sites, transmembrane regions), they supervise which residues each clause should match. The clauses themselves are noisy, the problem that DELE addressed in text classification: DELE uses bidirectional sentence-to-label attention to keep only the parts of a dictionary definition that fit the input. Our per-clause minimum serves a similar purpose, because irrelevant facets of a long description become separate clauses that can be left out of $A_F$ instead of diluting a single description vector.

We train with three losses. A contrastive loss pairs proteins with their own clauses, as in ProtST and ProtoCol. A localization loss rewards clause tokens that land on annotated residues. A hard-negative loss uses synthetically truncated or motif-deleted versions of real family members, which teaches the model what an incomplete domain looks like. Training on descriptions rather than on labeled examples follows label-description training, which reports average zero-shot gains of 17.5–18.9 points for RoBERTa on text classification. That paper supplies the method's logic but no biological evidence, so we treat its gains only as motivation.

The end-to-end use is a drop-in replacement for stage two of the standard pipeline. Candidates come from HMMER, BLASTP, or ProtoCol retrieval. Our verifier then replaces CD-Search filtering and returns a per-clause residue map alongside each decision.

Baselines

The naive reference point is the published two-step pipeline itself: HMMER and BLASTP screening followed by NCBI Batch CD-Search, Pfam, or SMART domain checks, as run in the pomegranate PYR/PYL study, the cucumber B-box study, and the rice KEA study. For held-out families we must hide their own Pfam profiles to simulate an uncovered family, and the pipeline then falls back to the nearest available profile. It counts as beaten if our verifier accepts more true remote members at equal or lower fragment-acceptance rate.

The strongest existing competitor is ProtoCol retrieval seeded with known family members, followed by HMMER-based domain verification. We run it from the released ProtoCol code with the same ESM-2 35M backbone, the same training proteins, and the same tuning budget. It is the strongest competitor because it already reaches a capped Recall@1 of 0.946 on SCOPe superfamilies (ProtoCol). It is also the natural alternative to text: nearest-neighbor similarity to seed sequences. For the Pfam clan split, no reported result was found on Lacuna for ProtoCol's numeric score, so we rerun it. Beating it means a higher precision of intact members at matched recall on held-out clans, with the gain concentrated on synthetic fragments and below-threshold remote members.

The pooled text baseline is ProtST used as a zero-shot verifier. A candidate is scored against the full family description by cosine similarity of pooled vectors, and the backbone is retrained on our clause data so the training data match. No reported result was found on Lacuna for ProtST on fragment rejection or gene family verification.

Three ablations isolate the key ingredients. Replacing the clause minimum with a single whole-description late-interaction score tests whether splitting the definition into clauses matters. Replacing text clauses with exemplar residue spans tests whether text is needed at all; the spans come from seed proteins at the same annotated positions. Removing the hard-negative fragments tests whether completeness is learned or simply inherited from contrastive training.

Why not just…?

Why not just keep HMMER plus CD-Search? On families that Pfam covers well, it will likely tie or win, and we expect to report that. The case for change is the remote tail. Hamamsy et al. report Needleman–Wunsch alignment at F1 0.098 versus DeepBLAST's 0.265 on Malidup twilight-zone duplicates. Candidates in that regime are exactly those that profile-based verification cannot confirm. The held-out-profile experiment measures how much of this tail our verifier recovers without admitting fragments.

Why not just use ProtoCol with seed sequences and skip text? This is the real threat. Whenever good seeds exist, residue-level similarity to seeds may verify as well as text does. The exemplar-span ablation settles the question directly. If exemplar spans match text clauses on held-out families, we will narrow the claim to families with few or no characterized seeds, plus the cross-family transfer of shared clauses. Text can transfer in that setting and seeds cannot. We will also drop text from the recommended pipeline wherever seeds suffice.

Why not just use ProtST's pooled embeddings with better prompts? Pooling collapses the per-clause evidence into one number. ProtoCol shows that pooled representations lose about seven points of retrieval accuracy even within sequence-only matching, and TALE reports that global embeddings fail on motif-dependent molecular function. The whole-description and pooled baselines show whether the clause minimum, rather than text alone, drives fragment rejection.

Why not just prompt a large language model with the sequence? Calling it once per gene across a genome of tens of thousands of genes is expensive. Barot et al. also note that sequence-to-text generation produces fluent descriptions that still require expert checking. Our verifier outputs residue positions, which a biologist can check directly against an alignment.

Proposed Experiments

The main benchmark is a group-disjoint Pfam clan split, following ProtoCol's protocol, with descriptions and clauses attached to each family. We add a companion test set built from held-out-clan members by truncating them, deleting annotated motif positions, or splicing in out-of-frame segments. Metrics are precision and recall of intact members, the fragment false-acceptance rate, and, where Swiss-Prot site annotations exist, localization accuracy: the fraction of each clause's best-matching residues that fall within annotated sites. SCOPe superfamily retrieval is a sanity check that the shared encoder does not regress against ProtoCol's 0.946.

The decisive check comes in three parts, each tied to an ablation above. Clause-minimum late interaction must beat ProtoCol-plus-HMMER on fragment false acceptance at matched recall on held-out clans. It must beat the exemplar-span ablation on families with at most one seed. Its localization must exceed the pooled ProtST baseline, which has no localization by construction. If the first holds but the second fails, the contribution is clause-level verification, not text.

As a genome-scale case study, we rerun the full inventories from the four plant studies, holding each family's own profile out of the verifier: 12 PgPYL, 22 CsBBX, 8 OsKEA, and 104 PsbHLH from the Prunus sibirica bHLH study. We ask whether the verifier reproduces the curated lists, and whether its disagreements are either fragments the authors excluded or plausible members worth expert review. OsKEA3, with its unusual TrkA_N and SMC_N domains, is a planned stress test. It should pass the required transporter clauses while its extra domains appear as unmatched regions instead of causing rejection.