One Epitope, Many Definitions: Definition-Invariant Training for Protective Antigen Identification

When a laboratory decides which patches of a pathogen's surface an antibody actually grips, it must turn continuous physics into a binary label: a residue counts as part of the epitope only if some antibody atom lies within a chosen distance cutoff. Different databases and different primary studies choose different cutoffs, typically between 4 and 6 ångströms, and never report the choice in a machine-readable way. The result is a quiet replication crisis inside protective antigen identification: models trained under one labeling rule are evaluated, fine-tuned, or stacked against data labeled under another, and nobody knows how much of the apparent failure to generalize is biology versus bookkeeping. This is precisely the pathology that clinical meta-analysis has documented for decades — the sarcopenic obesity literature, for example, pools to a hazard ratio of 1.21 overall, but the estimate swings from 1.18 to 2.54 depending on which diagnostic definition each cohort used (Association of sarcopenic obesity with the risk of all-cause mortality...).

This proposal treats study heterogeneity not as an application domain but as the core diagnostic lens for epitope prediction. The plan is to first measure how much of the disagreement between epitope models and across benchmark datasets is attributable to label-definition drift rather than to genuine differences in binding biology, and then to train predictors that are invariant to that drift: models that predict a continuous contact intensity from which any study's binary definition can be recovered by a simple threshold, with per-dataset random-effect offsets estimated in the spirit of random-effects meta-analysis. The payoff is a protective-antigen screener whose scores mean the same thing regardless of which labeling protocol produced the training evidence — a prerequisite for the multi-study, multi-tool vaccine pipelines that currently consume predictions from incompatible tools side by side (Epitope-Based Vaccines: The Next Generation of Promising Vaccines Against Bacterial Infection).

Core Claim

The claim is twofold, and each half is testable. First, a measurable fraction of the so-called "Generalization Wall" — the collapse of epitope predictors on unseen epitope groups reported by AsEP — and of the disagreement between benchmark studies is label-definition heterogeneity, not antigen biology. Second, replacing binary residue labels with a continuous distance-decay target, conditioning the model on the labeling protocol, and estimating study-level random effects will cut cross-definition performance degradation substantially while matching or exceeding the strongest existing predictor in-distribution. If the first half is wrong — if definition drift turns out to be a negligible source of variance — the audit itself is a publishable negative result that licenses the field's current practice, and the modeling workstream becomes moot.

Background and What Is Already Done

Three lines of work shape the gap. SEMA is closest to this proposal: it acknowledges boundary ambiguity by introducing an inner cutoff $R_1$ and an outer shell $R_2$, evaluates masked versus unmasked test sets, and even predicts continuous residue-antibody contact numbers, reaching a ROC AUC of 0.76 on an independent retrospective test set of structures deposited after January 1, 2020. But SEMA treats ambiguity within a single labeling protocol; it never trains or evaluates across protocols, and it explicitly leaves host-to-host and assay-to-assay variation unaddressed. AsEP reframes epitope prediction as bipartite link prediction between antibody and antigen graphs, curates 1,723 complexes clustered into epitope groups to prevent leakage, and shows that general surface-patch predictors fail catastrophically at the antibody-specific task (EpiPred MCC 0.029, MaSIF-site 0.037, versus 0.305 for its WALLE model) — but its clusters address data leakage, not label drift, and its "Generalization Wall" on unseen epitope groups is documented rather than explained. CHIMERA-Bench diagnoses exactly this kind of methodological heterogeneity for antibody design — fragmented tasks, unaligned database versions, divergent cutoffs — and resolves it by fiat, curating 2,922 SAbDab complexes under one unified task definition with three matched generalization splits. Standardization-by-curation, however, discards every study that does not conform, and cannot ingest new evidence labeled under a foreign protocol. This proposal differs from all three in direction: rather than removing heterogeneity from a benchmark, it models heterogeneity inside the learner, the way the federated analysis of incubation periods jointly fits individual-level data alongside heterogeneous summary statistics across outbreak cohorts (Federated analysis of incubation period distributions...).

Two further literatures supply method and motivation. The clinical side supplies the variance-accounting machinery: The Challenge of Heterogeneity in Medical Data Synthesis and the dual-antiplatelet meta-analysis after TAVR (Dual Antiplatelet Therapy vs. Single Antiplatelet Therapy...) — where observational pooling suggested a mortality benefit that vanished under randomized evidence — show what naive aggregation does to a signal. The immunology side supplies the consequence: SEPPA 3.0 demonstrates on 106 external glycoprotein antigens that N-glycosylation shifts which residues are antibody-accessible (external AUC 0.749 versus 0.740 on general proteins), a second, biochemical axis of heterogeneity that current protein language models in SEMA ignore entirely.

Research Program

Workstream one is a heterogeneity audit. Starting from the AsEP corpus of 1,723 antibody-antigen complexes, I will relabel every residue under a grid of heavy-atom distance cutoffs (4, 5, and 6 ångströms) and, where structures carry glycan annotations, under masked-glycan versus bare-backbone conditions. For each cutoff I will retrain or re-evaluate a fixed architecture and decompose performance variance into components attributable to model, labeling definition, and epitope group, reporting a between-definition heterogeneity statistic in the spirit of the $I^2$ index used in the sarcopenic obesity synthesis, where stratification resolved $I^2 = 64.3\%$ into definition-driven subgroups. This audit answers the question the field has not asked: how much of what we call generalization failure is definitional?

Workstream two builds the definition-invariant model. The architecture follows the strongest known recipe — protein language model embeddings combined with graph neural networks over the complex, as in WALLE — but the training signal changes. Instead of a binary label at a fixed cutoff, each antigen residue $i$ receives a soft target

$$y_i = \exp(-d_i / \sigma),$$

where $d_i$ is the minimum distance between any heavy atom of residue $i$ and any antibody atom, and $\sigma$ is a scale learned during training. A study labeling at cutoff $R$ then reads out positives as residues with predicted intensity above $\exp(-R/\sigma)$, so one model serves every protocol and no training data need be relabeled or discarded. On top of this, each source dataset contributes a learned scalar offset — a random effect — capturing its systematic label or assay bias, estimated jointly by an empirical-Bayes step analogous to DerSimonian-Laird, which lets the model consume heterogeneous evidence without forcing it onto a single schema. The design deliberately mirrors the choice PyHealth made for clinical task definitions, where a standardized five-stage pipeline with concept mapping made cross-database models comparable (PyHealth: A Deep Learning Toolkit for Healthcare Applications), but moves the harmonization inside the model rather than imposing it on the data.

Workstream three evaluates. The central question is cross-definition transfer: train under one cutoff, test under another, and measure how much degradation soft targets, protocol conditioning, and random effects remove relative to binary-label training. The second question is whether definition-invariant training also shrinks the unseen-epitope-group gap on AsEP's epitope-group splits, since if part of that gap is definitional it should be recoverable. The third question is robustness to the biochemical axis: whether the model trained on mixed glycosylated and bare antigens holds up on SEPPA 3.0's external glycoprotein set, where PTM-aware features previously mattered.

Proposed Experiments

The experiments run on three benchmark resources documented in the literature file: the AsEP dataset of 1,723 complexes with its epitope-group splits, evaluated by Matthews Correlation Coefficient following its convention; the SEMA retrospective test set of post-January-2020 structures with no training homologs, evaluated by ROC AUC; and SEPPA 3.0's external sets of 130 general protein antigens and 106 glycoprotein antigens, evaluated by AUC and balanced accuracy. The principal metrics are MCC for residue-level classification, ROC AUC for threshold-free ranking, and a new cross-definition degradation ratio — performance under a foreign cutoff divided by performance under the training cutoff — which the audit will establish as a meaningful quantity. Decisive checks are described in the closing section. All models, including baselines, will be trained on identical pooled corpora with matched tuning budgets and identical epitope-group and temporal splits, so that any difference is attributable to the labeling treatment alone.

Baselines

The naive reference point is pooled binary training: a WALLE-style model trained on all available complexes lumped together with one fixed 4.5 ångström label and no protocol information. This is the "do nothing" that the dual-antiplatelet and sarcopenic meta-analyses warn against, and it isolates what heterogeneity-awareness buys over aggregation alone. If pooled binary training already transfers across cutoffs, the core premise fails.

The strongest existing competitor is WALLE itself, the hybrid language-model-plus-graph model introduced by AsEP, which reports an MCC of 0.305 on its benchmark — the best number in the literature file — against 0.262 for AlphaFold2-Multimer, 0.037 for MaSIF-site, and 0.029 for EpiPred. WALLE will be retrained from its released recipe on our pooled corpus with the same compute and tuning budget as our model, so the comparison isolates the labeling treatment rather than architecture or data volume. Beating it means two things jointly: matching or exceeding MCC 0.305 on the standard AsEP splits under its own evaluation, and cutting the cross-definition degradation ratio by at least half relative to WALLE retrained identically. Matching WALLE in-distribution while winning cross-definition transfer would still count as a win, because in-distribution performance under one protocol is not the deployment condition.

A second comparator is SEMA, which reports ROC AUC 0.76 on its retrospective post-2020 test set and handles boundary ambiguity with its $R_1$/$R_2$ masked evaluation (SEMA); I will run SEMA's masking strategy as an alternative ambiguity treatment on the same cross-cutoff splits, so the proposal's continuous targets are compared against the best existing boundary-handling scheme rather than a strawman. For the glycoprotein axis, SEPPA 3.0's reported external glycoprotein AUC of 0.749 with balanced accuracy 0.665, and its false-positive reduction from 0.34 to 0.25 via spatial calibration (SEPPA 3.0), set the reference for PTM-aware performance.

The key ablation is our own architecture with binary fixed-cutoff labels, no protocol conditioning, and no random effects — identical network, identical data, identical tuning. Any gap between this ablation and the full model is attributable specifically to the heterogeneity-aware labeling treatment, which is the proposal's one new ingredient.

Why not just…?

Why not just standardize on one definition, the way CHIMERA-Bench did? CHIMERA-Bench (CHIMERA-Bench) proved standardization-by-curation works for antibody design, and it is a real threat to this proposal's motivation. But standardization is a benchmark-side fix that discards nonconforming evidence and cannot absorb new studies labeled under foreign rules; the public repositories keep growing under many protocols, exactly the situation that motivated PyHealth's harmonization layer for hospital databases. The decisive experiment is in the plan: retrain a single-cutoff standardized model and evaluate it under foreign cutoffs. If it does not collapse, I will report that standardization suffices and the modeling claim is dead.

Why not just mask the ambiguous residues, as SEMA already does? SEMA's masked evaluation excludes residues in the ambiguity shell between $R_1$ and $R_2$, which handles fuzziness within one protocol but cannot make a model trained at 4 ångströms comparable to labels at 6, and it discards boundary residues that carry most of the contested signal. SEMA is included as a head-to-head comparator on the cross-cutoff splits precisely so this alternative is settled empirically rather than by argument; if masking matches continuous targets under definition transfer, masking wins on simplicity and I will say so.

Why not just pool all the datasets and train a bigger model? The literature gives concrete evidence against scale-as-fix: CHIMERA-Bench found that smaller equivariant graph networks led larger diffusion models on sequence recovery and structural accuracy, indicating inductive bias beats parameter count (CHIMERA-Bench); the sarcopenic meta-analysis showed pooling without stratification conceals drivers varying from HR 1.18 to 2.54 (sarcopenic obesity meta-analysis); and the TAVR analysis showed pooled observational data asserting a mortality benefit that randomized evidence erased (Dual Antiplatelet Therapy...). Still, pooled big-model training is exactly the "do nothing" baseline above, run with the same budget, so the question is adjudicated by the experiment and not by analogy.

Why not just dock everything with AlphaFold2-Multimer? On AsEP, AlphaFold2-Multimer reached an MCC of 0.262 — below WALLE's 0.305 — while requiring 1.66 hours per prediction against seconds for WALLE (AsEP), which rules it out for proteome-scale screening, and it does not address label heterogeneity at all since it produces structures, not study-comparable residue scores. It is therefore a supplementary baseline on small test subsets only.

Decisive Checks

Four checks decide the proposal. First, the audit must show cross-definition degradation is real and large: fixed-architecture models trained at one cutoff and tested at another should lose a substantial, quantified fraction of MCC, and the variance decomposition should attribute a reportable share of benchmark disagreement to definitions. Second, the continuous-target model must cut that degradation by at least half against identically trained binary-label controls, across all six train-test cutoff pairings. Third, on AsEP's unseen-epitope-group splits, definition-invariant training should narrow the Generalization Wall gap relative to WALLE retrained on the same data; if the wall is untouched, the wall is genuinely biological and the audit will have cleanly separated the two causes. Fourth, an injected-noise test — synthetic datasets with known label bias per source — should show the estimated random effects recover the injected biases, validating the meta-analytic component. A positive result on all four gives the field, for the first time, epitope predictions whose scores are comparable across studies by construction; a negative result on the first check still retires a widespread unexamined assumption about why these models fail.