Finding proteins that share a distant evolutionary ancestor is hard once their amino-acid sequences have drifted to under about 30% identity, the so-called twilight zone, even though their 3D shapes and functions are often still conserved. The strongest recent neural approach, ProtoCol, skips the usual step of compressing a whole protein into one vector. It keeps one vector per residue and scores a query against a database protein by letting each query residue find its best-matching residue, a scheme called late interaction. This preserves local structural motifs and gives much better recall. It also multiplies database storage by the protein length, and the ProtoCol authors themselves identify that storage as the obstacle to searching UniProt-scale collections.
We propose to shrink the number of stored vectors rather than only their width. A small learned layer, a strided depthwise separable 1D convolution, merges each window of neighboring residue embeddings into one vector. "Depthwise separable" means the layer first filters each embedding channel along the sequence independently and then mixes channels with a cheap per-position projection. The compressed format is chosen together with the retrieval kernel: fixed stride, width aligned to matrix-multiply tiles, and 8-bit storage. This turns ragged per-residue scoring into regular dense matrix products. The claim is that such a layer, trained contrastively, can cut stored bytes per protein by roughly an order of magnitude while keeping twilight-zone recall on superfamilies never seen in training. The layer must also clearly beat the obvious alternative of averaging windows, which is the test that decides whether the idea has value.
Core Claim
ProtoCol showed that pooling a whole sequence into one vector destroys the signal needed for remote homology. A 35M-parameter late-interaction model reaches Capped Recall@1 of 0.946 on SCOPe and beats a pooled 650M ESM-2 model by nearly 7 points. The open question is where between "one vector per residue" and "one vector per protein" the useful information actually lives.
Our hypothesis is that the embeddings of a protein language model (a transformer pretrained to predict masked amino acids) are already contextual. The twilight-zone analysis of protein language models found that such embeddings carry substantial structural information, so long-range contacts are largely encoded before any compression happens. If that is right, merging a short window of adjacent residues loses little. A learned merge can also do better than an average. Its per-channel filters can learn, for example, to emphasize a helix-capping residue or a gap-prone loop boundary instead of blurring them.
This addresses a worry raised by work on the local correlation assumption, which argues that convolutional inductive biases succeed because image patches correlate locally with labels, a property protein sequences violate. We never ask the convolution to discover non-local structure. We only ask it to summarize locally something the transformer has already contextualized. That premise is testable, and the experiments below test it directly.
What is new is threefold. First, depthwise separable convolution is used as a sequence-length compressor for multi-vector retrieval, where prior use of these layers has been either for image FLOP reduction (Xception) or as a feature extractor in translation models (separable convolutions for machine translation). Second, the compressed format is designed around the scoring kernel's memory behavior. Third, the evaluation is organized around generalization to unseen superfamilies rather than in-distribution accuracy.
Research Program
The compressor. The encoder is ProtoCol's ESM-2 35M backbone and InfoNCE contrastive loss, reused from the official implementation. On top of it we add a compression head. The head has a depthwise 1D filter of width $K$ applied per channel with stride $s$, followed by a pointwise projection from the model's hidden width to a smaller width $d$, and then normalization.
A protein of length $L$ is then stored as $\lceil L/s \rceil$ vectors of width $d$ instead of $L$ vectors of full width. The query side can use the same head or stay uncompressed. We test both settings, because an asymmetric design keeps full query resolution at no storage cost.
The scoring kernel. The co-design component is deliberately modest and measurable; no new silicon is built. It follows the lesson of MP-Rec, which gained large speedups by matching a representation's memory profile to the hardware it runs on, and of the GPU and DCU studies of depthwise kernels (GPUs, DCU). Those studies found that FLOP savings vanish unless memory access is regular and operations are fused.
Our design makes three choices for this reason. A fixed stride turns variable-length proteins into blocks of predictable size. Choosing $d$ as a multiple of the tensor-core tile width, with 8-bit storage, makes the best-match scoring step a batched dense matrix product followed by a row-max. The query-side compression head is fused with scoring in a single kernel.
We will build a simple roofline cost model, which predicts time as the larger of compute time and memory-transfer time, for the scoring step. We then check its predictions against measured GPU latency and a CPU SIMD implementation. The survey AI+HW 2035 warns that analytical cost models often mispredict real hardware, so we report where our model is wrong.
The generalization question. Compression is trained on some superfamilies and evaluated on others. The risk is that the learned filters specialize to the motif vocabulary of the training folds and silently fail on new ones, a failure that averaging cannot have.
To detect this, we use ProtoCol's group-disjoint SCOPe and Pfam-clan splits, bin test pairs by maximum sequence identity following the thresholds of the twilight-zone study, and look specifically at the lowest bins. A secondary benefit is also tested. At fixed memory, compression allows sequences longer than ProtoCol's 256-residue truncation. We evaluate on the multi-domain scenario of the remote homology benchmark suite, where decomposing proteins was reported to boost AI methods by over 30%.
Baselines
Uncompressed ProtoCol is the strongest existing method and the ceiling. It is ProtoCol retrained from the released code on our exact splits, with the reported SCOPe Capped Recall@1 of 0.946 and roughly 24 ms per query on a GPU (ProtoCol). It competes directly because any compression can only trade accuracy away from it. All variants share its backbone, its training data, its tuning budget, and the same hardware for timing. Success means staying within one point of its Capped Recall@1, and within its seed-to-seed variance on the lowest identity bin, while storing at least 8× fewer bytes per protein.
The pooled single-vector model is the "do nothing" reference point: mean-pooled ESM-2, reported by ProtoCol as nearly 7 points below it on SCOPe. The paper gives the gap as a phrase, so we rerun it rather than infer an exact number. It marks the other end of the storage–accuracy curve, and our method should sit well above the straight line joining the two.
Classical search consists of MMseqs2 and HHblits. They are the practical incumbents. MMseqs2 is reported at over 1,000 ms per query in ProtoCol's setting, and HHblits was found to beat language-model methods on AUPRC, the area under the precision–recall curve (twilight-zone study). No numeric AUPRC value for HHblits is reported in our sources. We therefore report AUPRC for all methods, since our approach could plausibly worsen false positives. Beating HHblits on AUPRC is not required, but we must not widen the gap relative to uncompressed ProtoCol.
Stride-$s$ window averaging plus a linear projection is the critical baseline. It has the same storage as our method but replaces the learned depthwise filter with a fixed average. It is also trained end to end with the same budget and the same projection width.
Ablations isolate each ingredient: dimension reduction only ($s=1$), stride only (no width reduction), learned versus fixed depthwise filter, symmetric versus asymmetric query compression, and 8-bit versus 16-bit storage.
Why not just…?
Why not just shrink the vector width, as ColBERT-style systems already do with a linear projection? Reducing width with a per-position projection is exactly our layer at stride 1, so it is included as an ablation. Width reduction alone cannot remove the factor-of-$L$ storage blowup that ProtoCol identifies. Only reducing the number of vectors does that. The stride-only and width-only ablations will show which axis carries the redundancy.
Why not just average windows of residues instead of learning a filter? This is the real threat, and if it matches our method, the convolution is unnecessary. In that case we would report fixed-window pooling combined with the tile-aligned kernel as the contribution. The decisive comparison is on the lowest identity bin of unseen superfamilies, where blurring motif boundaries should hurt most. We will also inspect the learned filters for any correlation with predicted secondary-structure boundaries.
Why not just quantize or product-quantize ProtoCol's per-residue vectors? Quantization compresses each vector, while our method compresses along the sequence, so the two stack rather than compete. We will run product quantization alone at a matched byte budget. If it matches our accuracy per byte, sequence-axis compression adds nothing. We will also run the two combined.
Why not just prefilter with a fast single-vector index and rerank with full ProtoCol? A cascade reduces compute per query, but the full multi-vector database must still be stored for reranking, so the storage problem remains. It also caps recall at whatever the single-vector prefilter retrieves, which ProtoCol showed is weak in the twilight zone. We will measure recall@100 of a pooled prefilter to quantify that cap.
Proposed Experiments
The main experiment traces storage against accuracy. Stride $s$ ranges over 1, 2, 4 and 8, and width $d$ over 128, 64 and 32. We report Capped Recall@1, Recall@10 and Recall@100 on group-disjoint SCOPe and Pfam clans, along with SCOP2, which the twilight-zone study found harder for all models. Everything is broken down by identity bin, with three seeds per setting. Systems measurements cover bytes per protein, per-query latency and queries per second on GPU and CPU. These are compared against the roofline predictions and against unfused, unaligned implementations of the same model, to separate the representation's contribution from the kernel's.
The generalization check compares, per identity bin, the recall lost by learned compression against the loss from window averaging. If the learned filter's advantage holds on training-like superfamilies but disappears on held-out ones, the method has overfit to fold vocabulary. We would report that as a negative result.
The premise check separates two sources of information. We apply the same compressor to embeddings taken from early transformer layers, which are less contextual. If recall collapses there but survives on final-layer embeddings, it supports the claim that the transformer supplies non-local context and the convolution only summarizes it.
Finally, we apply the method to CATH-S20 and to the multi-domain and low-confidence AlphaFold scenarios of the benchmark suite, with lengths raised to 1,024 residues at the same memory as 256-residue uncompressed ProtoCol. Intrinsically disordered proteins are reported as a stated failure case, since every method in that benchmark failed on them.
References
- ProtoCol: Late Interaction Retrieval for Protein Homolog Search
- In the twilight zone of protein sequence homology: do protein language models learn protein structure?
- The Implications of Local Correlation on Learning Some Deep Functions
- Xception: Deep Learning with Depthwise Separable Convolutions
- Depthwise Separable Convolutions for Neural Machine Translation
- MP-Rec: Hardware-Software Co-design to Enable Multi-path Recommendation
- Optimizing Depthwise Separable Convolution Operations on GPUs
- Optimizing depthwise separable convolution on DCU
- AI+HW 2035: Shaping the Next Decade
- Benchmarking protein sequence and structure search methods for remote homology detection