The most accurate fast method for finding distantly related proteins currently keeps one learned vector per amino acid, not one per protein. It scores a query against a database protein by letting each query residue find its best-matching database residue, which works as a soft alignment without filling a dynamic-programming table. This makes it far better than squashing a protein into one vector. It also makes the index enormous. A UniRef-scale database has tens of billions of residues, so the per-residue vectors cannot fit on a GPU. Streaming them over the CPU–GPU link (PCIe) is about two orders of magnitude slower than on-GPU memory. The existing system sidesteps both problems by truncating every protein to 256 residues and keeping a small database in GPU memory.
We propose to borrow a trick from long-document text retrieval. Special "landmark" tokens are inserted at segment boundaries inside a full-length protein. Their output vectors are trained to summarize each segment in context, and only these landmark vectors stay on the GPU. For a query, the landmarks predict which segments of which database proteins will supply the best residue matches. Only those segments' residue vectors are fetched across PCIe, overlapped with ongoing compute. The matched segment pairs then give a narrow band for an optional exact alignment. The claim is that this makes full-length, multi-domain homology search at database scales beyond GPU memory nearly as accurate as an unlimited-memory residue index. The bytes moved per query should scale with the number of homologous segments, not with total protein length.
Core Claim and What Is New
The closest work is ProtoCol. It ports ColBERT-style late interaction (keep token vectors, score with a sum of per-token maximum similarities) to an ESM-2 35M protein language model. It reaches Capped Recall@1 of 0.946 at SCOPe superfamily level against 0.878 for a mean-pooled 650M ESM-2, at about 24 ms per query versus over 1,000 ms for MMseqs2. ProtoCol motivates this proposal: it shows residue granularity matters in the "twilight zone" of under 20–35% sequence identity. But it truncates sequences to 256 residues and evaluates only an in-memory testbed.
Writing the late-interaction score as $\mathrm{Score}(Q,D)=\sum_{i\in Q}\max_{j\in D} E_{Q,i}\cdot E_{D,j}$, where $E_{Q,i}$ is the vector for query residue $i$ and $E_{D,j}$ the vector for database residue $j$, our approximation restricts the inner maximum to a fetched set of segments $S$:
$$\mathrm{Score}_S(Q,D)=\sum_{i\in Q}\max_{j\in \cup S} E_{Q,i}\cdot E_{D,j}, \qquad \mathrm{gap}=\mathrm{Score}-\mathrm{Score}_S \ge 0.$$
The gap is non-negative because restricting the maximum can only lower it. The gap is exactly zero whenever $S$ contains every database segment that "owns" some query residue's best match. The new ingredient is a landmark scorer trained to rank segments by how much of the query they own. The fetch decision is then a learned prediction of where the gap would come from, made at segment rather than protein granularity.
Prior Work and Its Role
The method comes from contextual sentence encoding. Landmark Embedding inserts <LMK> tokens at sentence boundaries of continuous text and trains them with a position-aware loss. It improves LLaMA-2-7B on LongBench by 8.8 points while retrieving about 2,190 rather than 15,500 tokens. DCSR similarly pulls contextual sentence vectors from boundary tokens in full passages. We reuse the boundary-anchor idea but give it a different job. Landmarks are not the final retrieval unit; they are a cheap predictor of where fine-grained matches live. AGRaME shows a single ColBERTv2 multi-vector index can rank sub-units such as sentences (NQ sentence Precision@1 rising from 27.9% to 36.8%). This supports scoring domains and motifs from one full-length representation. AGRaME assumes the index is resident and leaves transfer cost untouched. CroCoAlign constrains our design. Its contextual aligner needs windowing for long texts and reintroduces cuts at window edges. We therefore encode proteins longer than ESM-2's context with overlapping windows and place landmarks away from window edges.
The transfer machinery comes from LLM offloading. SpeCache keeps a 1–2 bit "scout" copy of the attention cache on GPU to predict which full-precision entries to prefetch, reporting up to 4.6× throughput. Exploiting Sparsity for Long Context Inference keeps the cache in a CPU-side nearest-neighbour index and fetches under 2% of tokens, with 100% Needle-in-a-Haystack success at 1M tokens. Our landmark index plays the scout role. Unlike SpeCache, its predictions are trained for the retrieval objective rather than borrowed from a quantized copy. Unlike Synk et al., the expensive search stays on the GPU, so we avoid the CPU index-search latency they identify as a limitation. Teraio's GPUDirect Storage path supplies an optional NVMe tier for databases that exceed host RAM.
On the alignment side, Block Aligner grows and shrinks SIMD-computed blocks along high-scoring paths. Its own limitation is that low identity forces blocks to grow toward dense computation. AGAThA and KegAlign solve GPU load imbalance for guided alignment but depend on character seeds that fail on divergent sequences. We supply the band from neural segment matches instead of character seeds. This is a secondary deliverable that makes hits interpretable. It is not the central claim.
Research Program
We fine-tune ESM-2 35M with ProtoCol's late-interaction contrastive objective on full-length chains. Landmark tokens are inserted at segment boundaries, and long chains are encoded with overlapping windows. Residue vectors are 128-dimensional, as in ProtoCol.
A second loss trains the landmark scorer. For each training query–target pair, we compute the exact ownership mass of each target segment: the summed best-match similarities of the query residues whose argmax falls in that segment. The landmark pair score is trained with a listwise ranking loss to order segments by this mass. We follow Landmark Embedding's exponential position weighting so that landmarks near true homologous spans are favored. Hard negatives come from Pfam clans different from the query's, following ProtSent's use of Pfam hard negatives.
We will compare three segmentations: fixed 32- or 64-residue windows, boundaries at predicted secondary-structure transitions, and annotated domain boundaries. The structure-aware option is motivated by ProtoCol's observation that its residue similarity maps form blocks aligned with helices and strands.
The serving system works as follows. Landmark vectors (int8, one per 64 residues) stay in GPU memory. Residue vectors sit in pinned host memory, or on NVMe via GPUDirect Storage. For a query batch, the GPU scores all landmarks and selects the top segments per candidate protein under a byte budget. It then issues asynchronous transfers while computing exact MaxSim on the previously fetched batch.
A back-of-envelope calculation shows why scale is the regime where this matters. Take roughly 60 million sequences of roughly 300 residues each, at 128 dimensions. A 2-bit residue index is then on the order of 500 GB, far beyond one GPU. A 64-residue int8 landmark index is on the order of 35 GB, which fits. These are our own estimates, not reported results.
Finally, for top hits, the owned residue pairs define anchor diagonals. Block Aligner runs a banded alignment over an embedding-derived similarity matrix along them and emits a residue-level alignment.
Baselines
The do-nothing reference points are MMseqs2 and mean-pooled ESM-2. ProtoCol reports MMseqs2 at over 1,000 ms per query on SCOPe, and mean-pooled ESM-2 650M at 0.878 Capped Recall@1 at superfamily level. We rerun both on every database size with the same query sets. This shows whether neural search still beats a sensitive classical tool once memory constraints are real.
The accuracy ceiling is ProtoCol itself, using its released code at gabriellecohn/ProtoCol. It reports Capped Recall@1 of 0.946 at about 24 ms per query on SCOPe superfamily. We run it twice. The first run is as published, truncated to 256 residues. The second is an "unlimited-memory oracle": our same full-length model, with every residue vector resident, on a large-memory multi-GPU node. All other systems use the identical backbone, training data, and hyperparameter search budget. Beating the oracle on accuracy is not the goal. Matching it within one point of Capped Recall@1 while running on a single GPU is.
The strongest competitor for the efficiency claim is a protein-level two-stage pipeline. ProtSent-style single-vector embeddings (contrastively fine-tuned pooled ESM-2) retrieve the top-$k$ proteins from an in-GPU index. The full residue matrices of those $k$ proteins are then fetched with the same asynchronous pipeline and reranked by exact MaxSim. This is the most natural engineering answer and shares all our transfer machinery. It therefore isolates the one thing we add: segment-granular, landmark-predicted fetching. We tune $k$ and our segment budget to equal PCIe bytes per query, then compare recall. We also compare bytes at equal recall. Beating it means equal or better recall with substantially fewer bytes, and lower end-to-end latency, once the database exceeds GPU memory. The win should be concentrated on long and multi-domain targets.
We also include naive offloading, which streams all residue vectors without prediction, as the transfer-bound floor. A compressed resident index is a further baseline, covered below. Ablations swap trained landmarks for mean-pooled segment vectors, compare segmentations, and turn off transfer–compute overlap.
Why not just…?
Why not just compress the residue index until it fits in GPU memory? This is the strongest threat at small and medium scale. AGRaME builds on ColBERTv2's compressed multi-vector index, and on SCOPe or Pfam such an index fits easily. If quantized residue vectors fit, our method has no transfer advantage. We will report that honestly, defining the crossover database size where resident compression stops fitting or loses more than one point of recall. Our estimate puts UniRef-scale 2-bit indices well past single-GPU memory. Landmarks and compression also compose: we will test compressed residue vectors behind landmark scouting.
Why not just retrieve whole proteins with a single vector and rerank? The protein-level two-stage baseline above is exactly this. It could match us on single-domain SCOPe, where the protein is roughly the homologous unit. Evidence it falls short elsewhere comes from ProtoCol: pooled vectors lose localized motifs, and pooling a long multi-domain chain should dilute a single shared domain further. That would hurt first-stage recall and inflate the bytes fetched per relevant hit. The length- and domain-stratified experiment decides this. If two-stage matches us there, landmarks earn their keep only through full-length encoding, and we will say so.
Why not just chop proteins into independent 256-residue chunks and run ProtoCol on each? Independent chunks lose cross-boundary context. Landmark Embedding's gains over chunked retrieval in text are the motivating evidence. We test it directly by indexing overlapping independent chunks with truncated ProtoCol, at matched index size, on the multi-domain benchmark.
Why not just use MMseqs2 as a prefilter and neurally rerank? KegAlign notes that seed filtering is the bottleneck for divergent sequences. Character seeds are exactly what fails in the twilight zone. We measure the recall ceiling imposed by an MMseqs2 prefilter on SCOPe superfamily pairs below 25% identity.
Proposed Experiments
Accuracy is measured on SCOPe superfamily-level retrieval (Capped Recall@1, @10, @100), the setting where ProtoCol reports 0.946. SCOPe-40 structural retrieval follows ProtSent's protocol; ProtSent reports only relative gains there (+19.9% Recall@1 for 150M), so we will reproduce absolute numbers. Pfam clan-level homolog retrieval is also included, where no reported result was found on Lacuna for ProtoCol's Pfam numbers.
We build a new multi-domain benchmark from full-length Pfam-annotated chains. Single-domain queries are searched against full-length targets, with ground truth defined by shared clan. Results are stratified by target length (under 256, 256–1,000, over 1,000 residues) and by the fraction of the target that the homologous domain covers.
For scale, we grow the database from SCOPe to Pfam full-length to a UniRef subset approaching host-memory limits. At each size we report query latency, PCIe bytes per query, GPU idle fraction, and the recall gap to the unlimited-memory oracle.
The decisive check has three parts. On long and multi-domain targets beyond GPU-memory scale, landmark scouting should stay within one point of oracle Capped Recall@1. It should also move clearly fewer bytes than protein-level two-stage at equal recall. And it should beat truncated ProtoCol on targets over 256 residues. If the first holds but the second does not, the transfer contribution collapses into the simpler pipeline and the paper becomes about full-length landmark encoding.
A secondary check covers the alignment stage. We measure how often the landmark-defined band contains the optimal path of full Smith-Waterman over the same embedding similarity matrix, and the speedup Block Aligner gains from that restriction.
References
- ProtoCol: Late Interaction Retrieval for Protein Homolog Search
- Landmark Embedding: A Chunking-Free Embedding Method For Retrieval Augmented Long-Context Large Language Models
- Sentence-aware Contrastive Learning for Open-Domain Passage Retrieval
- AGRaME: Any-Granularity Ranking with Multi-Vector Embeddings
- CroCoAlign: A Cross-Lingual, Context-Aware and Fully-Neural Sentence Alignment System for Long Texts
- SpeCache: Speculative Key-Value Caching for Efficient Generation of LLMs
- Exploiting Sparsity for Long Context Inference: Million Token Contexts on Commodity GPUs
- Cost-Efficient LLM Training with Lifetime-Aware Tensor Offloading via GPUDirect Storage
- Block Aligner: an adaptive SIMD-accelerated aligner for sequences and position-specific scoring matrices
- AGAThA: Fast and Efficient GPU Acceleration of Guided Sequence Alignment for Long Read Mapping
- KegAlign: optimizing pairwise alignments with diagonal partitioning
- ProtSent: Protein Sentence Transformers