Compressed Codes for Convergent Repertoires: Sub-linear Retrieval and Hard-Split Verification of TCR Binding

Unrelated people exposed to the same antigen often independently evolve the same or nearly the same immune receptors — a phenomenon immunologists call the convergent, or "public," immune response. Detecting this convergence computationally is now the backbone of repertoire-based disease tracking, vaccine monitoring, and neoantigen discovery. But the detection pipelines are hitting a wall on two fronts at once. First, modern representations of receptors — dense floating-point vectors from transformers like TCR-BERT and TCR2vec — make searching a new patient's repertoire (up to a million receptor sequences) against reference databases of known binders an exhaustive, linear-time comparison problem that cannot scale to population biobanks. Second, Grazioli et al. showed that the headline numbers of deep binding predictors are largely an illusion: on a "hard split" where test peptides never appear in training, models like NetTCR-2.0 and ERGO II fall from roughly 0.95 AUROC to about 0.55 — a coin flip — because they memorize which sequences belong to binder populations rather than what makes a receptor actually recognize an epitope.

This proposal argues these two failures share a root cause and a possible fix. The exact problem was solved a decade ago in a different field: video copy detection, where hundreds of millions of high-dimensional descriptors had to be searched in near real time. There, Jégou et al. compressed descriptors into product-quantization codes with precomputed lookup-table distances, and Qin et al. later showed that learning discrete binary codes directly — rather than hashing after the fact — reduced memory 2,000-fold (180 MB of dense features to 90 KB of 128-bit codes on UCF101) while improving retrieval accuracy (59.71% versus 50.20% mean average precision for the continuous-relaxed SePH baseline). Compression done as part of learning forced the representation to keep only what mattered. We propose to bring that lesson to immune repertoires: learn compact binary and product-quantized codes over frozen transformer receptor embeddings, trained with a pairwise "same-binding / not-same-binding" verification objective, and test whether the compression both enables sub-linear population-scale retrieval of convergent receptors and restores generalization to unseen peptides.

Core Claim

The claim has two halves, one engineering and one scientific. The engineering claim: receptor embeddings from existing models such as TCR2vec (Jiang et al.), which reported AUROCs of $0.799 \pm 0.004$ (SVM) and $0.805 \pm 0.003$ (MLP) on 100,112 TCR–epitope pairs, can be compressed into 64–256-bit binary codes and short product-quantization codes that (a) preserve enough geometry for convergent-receptor retrieval and repertoire classification, and (b) cut memory and search latency by orders of magnitude using Hamming-distance or lookup-table scoring instead of dense Euclidean comparisons. The scientific claim: training the code with a pairwise verification loss — where the model must decide whether two receptors bind the same epitope, never seeing the epitope itself — acts as an information bottleneck that strips out the population-level biases responsible for the hard-split collapse, in the same way that Cinelli et al. found that filtering 8,000 triplet features down to roughly 12 relevant ones lifted SVM accuracy from 67% to 94% on mouse immunization repertoires rather than degrading it.

Formally, we learn a binary encoder $b(x) \in \{-1, +1\}^c$ over a frozen receptor embedding $x$, with a pairwise loss of the form

$$\mathcal{L} = \mathbb{E}_{(x_i, x_j, y)} \left[ y \cdot d_H\big(b(x_i), b(x_j)\big) + (1 - y) \cdot \max\big(0, m - d_H(b(x_i), b(x_j))\big) \right],$$

where $y = 1$ if receptors $i$ and $j$ bind the same epitope, $d_H$ is Hamming distance, and $m$ is a margin. Following Qin et al.'s anchor approximation, the codebook is trained against a few thousand anchor receptors rather than all pairs, keeping training linear in corpus size. The epitope never appears as input; only the pairing label does. This is the direct immune analogue of "same-action / not-same-action" verification on unseen classes, which Compressive Sequential Learning showed converges in 50 iterations versus 10,000 for competing dense methods while beating similarity baselines by 4.80% on ASLAN and 9.44% on KTH.

Research Program

Three components. First, we build the compressed index itself: take frozen TCR2vec (and, separately, TCR-BERT embeddings from the TCR-DeepInsight reference of Xue et al., which spans over 2 million single T cells across 46 disease conditions), and train binary codes and product-quantization codes on top. For retrieval we adopt asymmetric distance computation, following the INRIA copy-detection design: an uncompressed query receptor is compared against compressed database codes through precomputed lookup tables, giving sub-linear search with a memory footprint roughly three orders of magnitude below dense storage. Second, we train the pairwise verification head described above and evaluate it under the strict hard-split protocol of Grazioli et al. on their TChard dataset (over 528,000 peptide–CDR3β tuples from IEDB, VDJdb, McPAS-TCR, and MIRA). Third, we test the convergence-retrieval use case directly: given a newly sequenced repertoire, retrieve its convergent receptors from the compressed population reference, and measure whether retrieval reproduces the disease-associated public clusters that the dense pipeline found, at a fraction of the cost. For antibody repertoires, we apply the same machinery to Parapred-based paratope features, replacing the exhaustive within-length-bin all-against-all matching of Richardson et al.'s paratyping (84% precision versus 83% for clonotyping on Pertussis toxoid) with quantized approximate search — their method's $\mathcal{O}(N^2)$ bottleneck is precisely what an index exists to remove. Synthetic stress tests with tunable witness rates will use immuneSIM, whose simulated repertoires reproduce input germline usage at Spearman $r \ge 0.985$ and up to 99% productive in-frame sequences.

Baselines

The "do nothing" reference point is random guessing on unseen peptides, which Grazioli et al. report as approximately 0.55 AUROC for NetTCR-2.0 and ERGO II on the TChard hard split; any system that does not clear roughly 0.6 on that split has learned nothing transferable. The strongest existing method depends on the task. For sequence-level binding prediction, the strongest competitor is TCR2vec with a downstream MLP, which reported $0.805 \pm 0.003$ AUROC on its 100,112-pair benchmark (Jiang et al.); we will run it on TChard under the hard split ourselves, since its paper reports only the standard split — no reported hard-split result for TCR2vec exists in the research file. For repertoire-level classification, the strongest competitor is DeepRC (Widrich et al.), which reported a ROC-AUC of 0.831 on the real-world CMV cohort, against 0.825 for an SVM with a MinMax kernel and 0.699 for the classical Burden test; we will compare our compressed-bag classification against all three on the same cohort, giving DeepRC the same training budget and instance counts, and only replace its dense instance storage with our quantized codes. For the mouse immunization task, the reference is the 1-DBC + SVM pipeline of Cinelli et al. at 94% accuracy on CFA versus CFA+OVA. Fairness throughout means identical data splits, identical frozen backbone embeddings, identical tuning budget (grid over code length, margin, and codebook size), and identical hardware for all latency and memory measurements. The key ablation isolates the proposal's central ingredient: the same verification-trained encoder kept at full floating-point precision, with no binarization or quantization. If the dense ablation matches the compressed model's hard-split AUROC, then compression is only an engineering win; if it does worse, the discrete bottleneck itself is responsible for generalization, which is the more interesting outcome. A second ablation replaces the learned codes with post-hoc hashing of TCR2vec embeddings (no verification loss), separating the contribution of the training objective from the contribution of discreteness.

Why not just…?

Why not just count k-mers or triplets and match strings? The evidence says this fails twice. Unfiltered 8,000-dimensional triplet counts drove an SVM to only 67% accuracy in Cinelli et al. — the noise floor that motivated their filtering — and Grazioli et al. showed sequence-correlation models collapse to roughly 0.55 AUROC on unseen peptides. Richardson et al. add that exact matching misses functional convergence across germlines entirely: their paratype-unique binders, invisible to clonotyping, had a 30% wet-lab-confirmed hit rate. Our experiments include the triplet-SVM baseline on every dataset precisely to quantify this gap.

Why not just use PCA on the transformer embeddings? This is a real threat, and we treat it as such rather than dismissing it. The dramatic collapse evidence — PCA dropping covariance-descriptor accuracy from 88.9% to 33.8% in Dai and Srivastava) — comes from structured SPD matrices, not from transformer embeddings that were trained to be Euclidean in the first place; indeed TCR2vec's similarity-preservation objective was designed to make embedding distance track sequence similarity. It is entirely possible PCA to 64 dimensions costs almost nothing. We therefore run PCA as a first-class baseline on every task, and the anchor ablation (PCA versus our codes at matched dimensionality) will settle it empirically. Our claim survives in weaker form if PCA matches accuracy: it still cannot deliver discrete Hamming search or 2,000-fold storage reduction, which is the scalability half of the proposal.

Why not just keep dense transformer embeddings and brute-force search? This works today at the scale of a single study but not at the scale the field is heading toward. TCR-DeepInsight already handles 2 million cells with uncompressed continuous vectors and standard clustering (Xue et al.), and the authors of DeepRC explicitly note that dense instance storage forces them to truncate gradient updates to the top-10% of sequences (Widrich et al.) — an arbitrary cutoff our quantized bags would remove. More importantly for the scientific half, dense embeddings trained without a pairwise objective demonstrably memorize: hard-split AUROC near 0.55 (Grazioli et al.). Our dense-ablation baseline makes this comparison direct.

Why not just predict 3D structures for everything? Latency rules it out. RosettaAntibody needs hundreds of CPU hours per sequence, and even fast ABodyBuilder needs about 30 seconds per sequence (Kovaltsuk et al.); a repertoire of a million receptors is months to years of compute per patient. Worse, the structures are unreliable exactly where it matters: TCRcost found raw AlphaFold2 predictions give only 0.375 binding accuracy on CDR3 loops (below chance) at 12.753 Å RMSD, needing a learned correction module to reach 0.762. Structural signals belong in our pipeline as features — Parapred-style predicted paratope residues, extracted in roughly 0.1 seconds per sequence (Richardson et al.) — not as full coordinate reconstruction.

Proposed Experiments

The decisive experiment is the TChard hard split: train the pairwise verification head on receptor pairs sharing epitopes, test on peptide–receptor combinations whose peptides never appear in training. Success means clearing the 0.55 memorization floor by a wide margin and approaching the 0.805 dense-split performance of TCR2vec — we set 0.70 AUROC as the pre-registered target for the compressed model. A secondary binding benchmark uses TCR2vec's 100,112-pair dataset under both standard and hard splits, so we can quote our numbers against their reported $0.799 \pm 0.004$ and $0.805 \pm 0.003$ directly. For repertoire classification, we use the CMV cohort, where DeepRC's reported 0.831 ROC-AUC (SVM MinMax 0.825, Burden test 0.699) is the bar; our compressed-bag variant must stay within a point or two while reporting exact memory footprint and wall-clock time, targeting at least a hundredfold reduction in instance storage, with the PRBC result of 180 MB to 90 KB as the field's proof that such reductions are achievable without accuracy loss. For biological fidelity, we run the 18-mouse CFA versus CFA+OVA task against Cinelli's 94% accuracy, checking that our learned codes recover triplet-level motif information (the R-motif work of Dahal-Koirala et al. and the triplet convergence results of Thomas et al. and Sun et al. tell us convergence lives in short local stretches, which is exactly the granularity binary codes should preserve). For convergence retrieval at population scale, we query held-out COVID and melanoma repertoires against the compressed TCR-DeepInsight reference and measure whether the disease-associated public clusters ($P < 0.001$ in the original dense pipeline) are recovered, reporting recall-at-k against exhaustive dense search as the gold standard, plus the compression factor and queries-per-second curve from one to a hundred million receptors. For antibodies, we benchmark quantized paratope retrieval on the Pertussis toxoid data (1,290 paired sequences, 84% precision for paratyping, 83% for clonotyping), testing whether approximate search retains that precision while eliminating the quadratic within-bin matching. Finally, scaling behavior itself is a deliverable: retrieval time should grow sub-linearly in database size while dense search grows as $\mathcal{O}(N \cdot D)$, and we will plot both curves on immuneSIM-simulated corpora up to $10^8$ receptors with implanted convergent motifs at witness rates down to 0.01%, the regime DeepRC already showed is detectable (0.984 AUC on synthetic data at 0.1% witness rate) but only with dense GPU storage.

References

  1. On TCR binding predictors failing to generalize to unseen peptides
  2. INRIA LEAR-TEXMEX: Video Copy Detection Task
  3. Binary Coding for Partial Action Analysis with Limited Observation Ratios
  4. TCR2vec: a deep representation learning framework of T-cell receptor sequence and function
  5. Feature selection using a one dimensional naïve Bayes' classifier increases the accuracy of support vector machine classification of CDR3 repertoires
  6. Compressive Sequential Learning for Action Similarity Labeling
  7. A pan-disease and population-level single-cell TCRαβ repertoire reference
  8. A computational method for immune repertoire mining that identifies novel binders from different clonotypes, demonstrated by identifying anti-pertussis toxoid antibodies
  9. immuneSIM: tunable multi-feature simulation of B- and T-cell receptor repertoires for immunoinformatics benchmarking
  10. Modern Hopfield Networks and Attention for Immune Repertoire Classification
  11. Video-Based Action Recognition Using Dimension Reduction of Deep Covariance Trajectories
  12. How B-Cell Receptor Repertoire Sequencing Can Be Enriched with Structural Antibody Data
  13. TCRcost: a deep learning model utilizing TCR 3D structure for enhanced of TCR–peptide binding
  14. Comprehensive Analysis of CDR3 Sequences in Gluten-Specific T-Cell Receptors Reveals a Dominant R-Motif and Several New Minor Motifs
  15. Tracking global changes induced in the CD4 T-cell receptor repertoire by immunization with a complex antigen using short stretches of CDR3 protein sequence
  16. Specificity, Privacy, and Degeneracy in the CD4 T Cell Receptor Repertoire Following Immunization