A PROTAC (proteolysis-targeting chimera) is a two-headed small molecule: one end grabs a disease protein, the other end grabs an E3 ubiquitin ligase — the cell's own tagging machinery — and the resulting "ternary complex" gets the disease protein marked for destruction. Designing these molecules computationally has advanced fast, but the field's evaluation practice has not. Generative pipelines score candidates by surface similarity to known degraders (novelty and memorization rates against databases), or hand them to a black-box neural predictor that returns a single probability of degradation. Neither practice can say the thing that matters most: is this new molecule functionally equivalent to a known active degrader — for this target, with this E3 ligase — despite looking chemically nothing like it?
Text generation went through exactly this crisis and solved it. N-gram metrics like ROUGE and BLEU, which count overlapping words, fail in both directions: a perfect paraphrase with no shared vocabulary scores zero, while a "word salad" of jumbled keywords scores high. The fix was to evaluate functionally and semantically: decompose the output into atomic factual statements and verify each one (SEval-Ex: A Statement-Level Framework for Explainable Summarization Evaluation), or probe whether the candidate answers the same questions as the reference. This proposal imports that machinery into targeted protein degradation. Degradation is not one property of a molecule; it is a five-step causal chain, and we propose to evaluate degraders by whether they satisfy the same chain of functional "statements" as a reference degrader — an interpretable, per-stage verdict that chemical overlap metrics and scalar predictors structurally cannot provide.
Core Claim
Two PROTACs should be called functionally equivalent for a given target–ligase context if they satisfy the same functional statements: (1) binding the target protein of interest with usable affinity, (2) recruiting the E3 ligase, (3) assembling a ternary complex without steric clash, (4) orienting target surface lysines productively toward the ubiquitin-transfer machinery, and (5) presenting a degradable complex — even if the two molecules share zero chemical scaffold overlap. Conversely, two molecules differing by a single methylene unit or one inverted stereocenter should be called non-equivalent if that perturbation breaks any statement — a "chemical activity cliff," the direct analog of a word salad scoring high on lexical overlap. Formally, with $T = (\text{POI}, \text{E3})$ denoting the biological context and $v(D, T) \in \{0,1\}^{5}$ the vector of probe answers for degrader $D$:
$$ v(D,T) = \big(v_1,\dots,v_5\big), \qquad D \equiv_T D' \iff v(D,T) = v(D',T), $$
and the headline degradation score is an interpretable weighted aggregate $S(D,T) = \sum_k w_k\, v_k(D,T)$, with every weight and verdict inspectable. The context-dependence is essential and follows DegradeQuery: Counterfactual Tuple Pretraining for Context-Aware PROTAC Degradation Prediction, which showed the same molecule can be active against one target–E3 pair and dead against another; equivalence is always relative to $T$, never an intrinsic property of the molecule.
No existing work provides this. LM-PROTAC: a language model-driven PROTAC generation pipeline with dual constraints of structure and property tokenizes molecules into sub-fragments and generates candidates (novelty 0.997), but evaluates only with novelty scores and property filters — the lexical-overlap regime. CupOFLATTE: Coupled Objective-Guided Discrete Flows via Linker Assembly for Targeted PROTAC Engineering proved the point dramatically — linkers with 0.0% memorization against the training database were predicted 1.7x to 1.9x more active than baseline designs — but then outsourced the activity judgment entirely to DegradeMaster as an opaque oracle. Accurate PROTAC-targeted degradation prediction with DegradeMaster outputs one scalar probability; its attention-pooling visualization is not a per-stage causal verdict. The semantic equivalence literature — BLEURT: Learning Robust Metrics for Text Generation, T5Score: Discriminative Fine-tuning of Generative Evaluation Metrics, and SEval-Ex — has never been applied to structured chemical-biological languages. We claim the transfer is not cosmetic: the perturbation-pretraining trick that made BLEURT robust is precisely what a PROTAC evaluator needs to survive publication bias, and the atomic-statement decomposition that made SEval-Ex interpretable is precisely what PROTAC evaluation needs to localize failure.
Research Program
The first component is the functional rubric itself: a fixed, biologically grounded set of five statements per candidate, each answerable by an in silico probe. The probes are deliberately cheap. Binary binding (statements 1–2) uses sequence- and graph-based affinity prediction. Ternary geometry (statement 3) uses a PROflow-style learned structure predictor rather than physics docking: PROflow: An iterative refinement model for PROTAC-induced structure prediction predicts ternary complexes in roughly 44 seconds per molecule, up to 60x faster than physics-based PRosettaC, at 8.35 Å average interface RMSD and 0.264 fraction of native contacts on its pseudo-ternary benchmark — imperfect geometry, but sufficient for coarse clash and lysine-orientation checks (statement 4), which is all a re-ranker needs at the scale of ten thousand candidates, the volume LM-PROTAC generates.
The second component is the training signal, and this is where the lexical-overlap literature enters as mechanism rather than metaphor. Following BLEURT — which gained robustness against quality drift by pre-training on 6.5 million synthetically perturbed Wikipedia sentence pairs — we pre-train the probe encoder on chemically grounded perturbations of the 7,134 unlabeled tuples in PROTAC-8K: extend a linker by one methylene, rotate the attachment vector, swap a functional group for a bioisostere, invert a stereocenter, or swap the target/E3 partner. The training objective enforces the two invariances the semantic-equivalence gap demands: probe answers must be invariant to paraphrase-type perturbations that preserve the functional chain, and sensitive to cliff-type perturbations that break it. This directly upgrades DegradeQuery's counterfactual tuple pretraining, whose authors themselves flag that their random component-swapping leaves "hard" counterfactuals — near-identical variants that fail to degrade — unaddressed. Our cliff perturbations are exactly those hard negatives. Where DegradeMaster exploits unlabeled data through memory-based pseudo-labeling (a 3.3% AUROC gain, at the acknowledged risk of confirming early biases on ambiguous candidates), our perturbation regime supplies supervision without inventing labels.
The third component is the output: not a scalar but a per-stage verdict matrix in the spirit of SEval-Ex's True Positive / False Positive / False Negative decomposition — for a degrader, "which step of the causal chain does this candidate fail at, and against which reference degrader is it equivalent." This is what lets a chemist act on a rejection, and what no current tool offers.
Baselines
The naive reference point is nearest-neighbor chemical similarity: rank a candidate by its maximum fingerprint (Tanimoto) similarity to any known active degrader for the same target–E3 pair in PROTAC-DB 3.0: an updated database of PROTACs with extended pharmacokinetic parameters. This is the "do nothing" option because it is what current pipelines implicitly fall back on when they report novelty or memorization, and it requires no learning. It will be run on exactly the same PROTAC-8K splits as our model. Beating it is trivial on paper but diagnostic in practice: on our cliff test set (see below) its ordering should be near chance, and any metric that merely rediscovers similarity will fail there too. No reported result found on Lacuna for a Tanimoto nearest-neighbor baseline on PROTAC-8K, so we will establish its numbers ourselves under identical conditions.
The strongest existing competitor is DegradeQuery, which reports AUROC of 0.9065 and accuracy of 0.8500 on the labeled PROTAC-8K benchmark, outperforming DegradeMaster's 0.8825 AUROC; DegradeMaster in turn reports 88.33% accuracy on its BRD9 (VZ185) case study and 77.78% on KRAS-mutant (ACBI3) degradation. Both will be re-run on identical splits of the 1,502 labeled records with access to the same 7,134 unlabeled tuples and the same tuning budget (matched hyperparameter search count, matched GPU-hours), so the comparison isolates the evaluation scheme rather than compute. Beating DegradeQuery means AUROC above 0.9065 on the standard split — an honest stretch goal — or matching its AUROC while winning decisively on the two tests it was never designed for: the cliff set (high-overlap pairs with opposite labels) and the paraphrase set (low-overlap pairs with the same label), where we require large separation in equivalence scores where Tanimoto similarity provides none. We state this disjunctive success criterion up front because a scalar predictor could plausibly match our aggregate score while retaining none of the equivalence semantics; the claim we defend is the combination.
Two ablations isolate the proposal's ingredients. Removing the statement decomposition — collapsing the five probes into a single pooled score trained identically — tests whether interpretable decomposition costs or buys accuracy (SEval-Ex's evidence suggests decomposition itself buys correlation: 0.580 Spearman on SummEval consistency versus 0.115 for ROUGE-L). Replacing our chemically grounded perturbation pretraining with DegradeQuery-style random tuple swapping tests whether the perturbation design, not just extra unlabeled data, is responsible for any cliff-set gains.
Why not just…?
Why not just use fingerprint similarity to known actives? Because the entire DegradeMaster program documents its failure. DegradeMaster's ablations showed removing the 3D spatial encoding caused the largest performance drop — the "2D trap" — and the field's defining pathology is that a perturbation preserving a Tanimoto similarity above 0.90 with an active compound can abolish degradation entirely, while CupOFLATTE showed linkers with zero database overlap were predicted 1.7x to 1.9x more active than baselines. Similarity ordering is near-chance exactly on the pairs that matter for equivalence. Our cliff set will quantify this directly rather than assert it.
Why not just use DegradeMaster or DegradeQuery as the oracle, as CupOFLATTE already does? This is the real threat, and we take it seriously: DegradeQuery at 0.9065 AUROC is a strong scalar, and one could argue our aggregate score is just another scalar. Three things fall short, and our experiments are built to demonstrate each. First, a scalar cannot express an equivalence relation between two molecules, which is the object CupOFLATTE's 0.0%-memorization result implicitly demands and leaves unformulated. Second, a scalar cannot localize failure — DegradeMaster's own ablations show prediction quality degrades with docking inaccuracy, but a user cannot tell whether a low score means bad binding, bad geometry, or bad lysine orientation; our verdict matrix can, and the failure-localization experiment on known inactive near-misses tests whether those localized verdicts are actually right. Third, DegradeQuery's random counterfactual swapping is explicitly weak on hard negatives, which is precisely our cliff set; if DegradeQuery matches us there too, the interpretability claim alone would not justify the work, and we would report that honestly.
Why not just run physics-based docking or molecular dynamics? Latency and evidence both rule it out as the primary instrument. PROflow's authors document that physics-based ternary modeling is too slow at the scale generative pipelines demand (PROflow is up to 60x faster than PRosettaC), that only 18 ternary crystal structures exist in the PDB to anchor physics at all, and that accuracy degrades badly from unbound protein conformations. We instead use learned geometry as a probe inside a cheap rubric — a hybrid — and validate the probe's verdicts against the physics-anchored case studies DegradeMaster used (BRD9/VZ185, KRAS/ACBI3) as an external consistency check.
Why not just compute BLEU/ROUGE over LM-PROTAC's sub-molecule tokens? This is the chemical word-salad. SEval-Ex reports ROUGE-L correlating at only 0.115 Spearman with human consistency judgments on SummEval, against 0.580 for semantic statement verification — and chemistry amplifies the flaw: a SMILES string (the standard line notation for molecules) is non-unique, so the same molecule written from a different starting atom scores differently, and DeepPROTACs is a deep learning-based targeted degradation predictor for PROTACs already showed that representing the linker as a raw 1D string is a documented weakness. Token overlap inherits every pathology we are trying to remove.
Proposed Experiments
Experiment 1 is the standard benchmark comparison: train the rubric-augmented model on PROTAC-8K's 1,502 labeled records with perturbation pretraining on the 7,134 unlabeled tuples, and report AUROC and accuracy against DegradeQuery (0.9065 / 0.8500), DegradeMaster (0.8825), and the DeepPROTACs-style structural baseline (which reported 77.95% accuracy and 0.8470 AUROC on its own 2,832-sample curation, so we will re-evaluate it on the common split). Experiment 2 is the decisive semantic test: construct two curated pair sets from PROTAC-DB 3.0 — "PROTAC-Para" (molecules with low chemical overlap but the same experimentally observed activity against the same target–E3 pair) and "PROTAC-Cliff" (pairs with high overlap, including documented activity cliffs, with opposite labels). Success requires high equivalence scores on Para pairs and clean separation on Cliff pairs, with Tanimoto similarity run as the foil. This mirrors SEval-Ex's controlled-corruption protocol, where scores dropped from 0.970 to 0.328 under injected incorrect events; we expect an analogous collapse in our aggregate score under a single cliff perturbation, and — crucially — the verdict matrix should name which stage broke. Experiment 3 tests the generative-utility claim: re-rank CupOFLATTE's BRD9–VHL designs (where it achieved 72% assembly validity against a 0% diffusion baseline) and an LM-PROTAC-style candidate pool, measuring rank agreement with DegradeMaster's predicted activity, and adjudicating disagreements on the labeled BRD9 and KRAS case studies. Experiment 4 runs the two ablations above. The decisive final check, gated on the compute-gated success of the first four, is wet-lab: synthesize the top-ranked and deliberately bottom-ranked novel candidates — following LM-PROTAC's precedent, which took two of 10,000 candidates to validated dose-dependent Wnt3a degradation in a 50-day cycle — and verify that the per-stage verdicts predict the observed failure mode, not just the binary outcome.
The honest risk register is short. The equivalence relation is only as granular as the binary labels it is trained against, so coarse $DC_{50}$ thresholds in PROTAC-DB may blur genuine partial-equivalence; the cliff set construction is the mitigation, since cliffs are the regime where binary labels are most reliable. And if the strongest scalar baselines match us everywhere including the cliff set, the proposal reduces to an interpretability layer on DegradeQuery — an outcome the experiments are explicitly designed to detect early rather than argue around.
References
- SEval- [inline-graphic not available: see fulltext]: A Statement-Level Framework for Explainable Summarization Evaluation
- DegradeQuery: Counterfactual Tuple Pretraining for Context-Aware PROTAC Degradation Prediction
- LM-PROTAC: a language model-driven PROTAC generation pipeline with dual constraints of structure and property
- CupOFLATTE: Coupled Objective-Guided Discrete Flows via Linker Assembly for Targeted PROTAC Engineering
- Accurate PROTAC-targeted degradation prediction with DegradeMaster
- BLEURT: Learning Robust Metrics for Text Generation
- T5Score: Discriminative Fine-tuning of Generative Evaluation Metrics
- PROflow: An iterative refinement model for PROTAC-induced structure prediction
- PROTAC-DB 3.0: an updated database of PROTACs with extended pharmacokinetic parameters
- DeepPROTACs is a deep learning-based targeted degradation predictor for PROTACs