When a compound looks potent in a dish but fails in an animal, we usually blame biology: poor pharmacokinetics, missing immune surveillance, the wrong species. But there is a second, quieter explanation that the literature has never quantified. A large fraction of "active" compounds in public screening data are not modulating their supposed target at all — they are killing cells wholesale, forming invisible colloidal clumps that non-specifically inhibit enzymes, or tampering with the assay's readout chemistry. Two research communities document these artifacts in detail, and a third documents the systematic failure of in vitro hits in animals, but nobody has connected them: no study asks whether the compounds that fail in vivo are disproportionately the ones whose in vitro activity was an artifact in the first place.
This proposal makes that connection the object of study. The core claim is that a measurable part of the in vitro-to-in vivo translation gap — and part of the cross-species validation failure rate — is label noise: assay-interference artifacts baked into the activity labels used to nominate targets, train predictive models, and justify animal experiments. We will build an interference-attribution layer over existing detectors, use it to scrub training labels across public benchmarks, and test two falsifiable predictions: that scrubbed labels improve out-of-distribution generalization (the standard computational proxy for translation, since novel scaffolds are what animals and patients actually present), and that interference flags, computed purely from in vitro data, predict which published in vivo "efficacy" claims lack genuine target engagement. What survives the scrubbing should be the part of the gap that is genuinely biological — and we can bound that residual against known biological ceilings.
Core Claim
Assay interference — non-specific mechanisms like general cytotoxicity, colloidal aggregation (hydrophobic molecules clumping into sub-micrometer particles that soak up proteins), reporter inhibition, and redox cycling — inflates in vitro activity labels. Models and target-validation arguments trained on those labels inherit the noise. Three observations motivate the claim. First, Anne E. Carpenter, Srijit Seal, et al. showed that a single feature — the number of cell nuclei in an image, a proxy for general cell death — matches or beats deep neural networks on standard bioactivity benchmarks, because 32% of those assays are predicted by compounds simply killing cells (Counting cells can accurately predict small-molecule bioactivity benchmarks). Second, Yunchao Liu et al. showed that machine learning models trained on uncurated high-throughput screening data learn assay artifacts, and that rigorous wet-lab counter-screening fixes this (WelQrate: Defining the Gold Standard in Small Molecule Drug Discovery Benchmarking). Third, Kexin Huang et al. showed that drug–target interaction models collapse from a Pearson correlation of roughly 0.7 on random splits to roughly 0.4 on temporal splits — a collapse usually attributed to distribution shift in the molecules (Therapeutics Data Commons: Machine Learning Datasets and Tasks for Drug Discovery and Development). None of these studies asks the next question: if interference-driven false positives are time-stamped, scaffold-clustered, and unevenly distributed across assays, how much of that collapse — and how much of the in vivo failure rate — do they actually explain? We hypothesize the answer is a substantial, quantifiable fraction, and that removing it is cheap compared to the biology that remains.
The operational form of the claim is a relabeling rule. For a compound $i$ measured in assay $j$ with binary activity label $y_{ij}$, we compute an interference probability $p^{\mathrm{int}}_{ij}$ from existing detectors and replace the label with
$$\hat{y}_{ij} = y_{ij}\,\bigl(1 - p^{\mathrm{int}}_{ij}\bigr),$$
where $p^{\mathrm{int}}_{ij}$ is the maximum over mechanism-specific detectors: colloidal aggregation from the MEGAN graph attention network (Mitigating Molecular Aggregation in Drug Discovery With Predictive Insights From Explainable AI), luciferase inhibition, thiol reactivity, and redox activity from E-GuARD's mechanism-specific classifiers (E-GuARD: expert-guided augmentation for the robust detection of compounds interfering with biological assays), and general cytotoxicity from the cell-count residual of Carpenter et al. The detectors already exist and are individually validated; the new science is in the composition, the scrubbing policy, and above all the downstream test against in vivo and cross-species endpoints, which no interference tool has ever faced.
Research Program
The program has three workstreams. Workstream A builds the interference-attribution layer: for every compound-assay pair in the study corpora, we run MEGAN (82% test accuracy, 81% F1 on its aggregation benchmark, and — critically — 68% versus 50% for the ChemAgg XGBoost baseline on 58 external drugs, showing it transfers beyond training data), the E-GuARD mechanism classifiers (MCC up to 0.47 on thiol reactivity, redox activity, and luciferase inhibition), and a cytotoxicity flag derived from the cell-count baseline. The output is a per-pair interference posterior with a mechanism attribution, not a single bad-actor flag — because the mechanism matters downstream: an aggregator may be fixable by adding detergent to the assay, while a general cytotoxin should change how we interpret any phenotypic readout, as Dahlin et al. argue in their characterization of cellular injury reference compounds (Reference compounds for characterizing cellular injury in high-content cellular morphology assays).
Workstream B measures whether scrubbed labels close part of the out-of-distribution gap. We retrain standard models — deliberately simple ones: multi-layer perceptrons on 2D chemical descriptors, which Liu et al. and Huang et al. both showed match or beat graph neural networks — on raw versus scrubbed versions of the WelQrate suite (nine PubChem high-throughput screening datasets spanning GPCRs, ion channels, transporters, kinases, and enzymes, 66,000 to over 300,000 compounds each, with active rates of 0.039% to 0.682%) and the Cell Painting benchmarks analyzed by Carpenter et al. The evaluation metric is early enrichment (logAUC and BEDROC) under Bemis-Murcko scaffold splits, the setting in which WelQrate showed that random-split advantages evaporate. Because WelQrate also ships wet-lab-curated versions of the same datasets, we get a three-way comparison: models trained on raw data, on computationally scrubbed data, and on wet-lab-curated data. This tells us how much of the curation benefit is recoverable without touching a pipette, and — by difference — how much interference is invisible to our current detectors.
Workstream C is the decisive test against reality: a paired in vitro/in vivo corpus. We assemble, from the indexed literature, compound-target pairs where both in vitro activity claims and in vivo animal efficacy claims exist, and score each with the interference layer before reading out the in vivo outcome. The corpus includes natural product mechanisms claims of the kind that Gap 4 flags as unresolved: quercetin's ferroptosis inhibition in metabolic-associated fatty liver disease via p38 MAPK/ERK (The mechanism study of quercetin isolated from Zanthoxylum bungeanum maxim. inhibiting ferroptosis and alleviating MAFLD through p38 MAPK/ERK signaling pathway based on lipidomics and transcriptomics) — where quercetin is a textbook frequent hitter for redox cycling and metal chelation — curcumol's claimed SKP2-mediated G1 arrest confined entirely to static culture (Curcumol Induces G1 Phase Arrest in SK-Hep-1 Cells by Targeting SKP2-Mediated p27 Degradation), and matrine's mitochondrial-apoptosis myeloma kill without any in vivo tolerability data (Matrine induces apoptosis of human multiple myeloma cells via activation of the mitochondrial pathway). For the in vivo side we code two things separately: phenotypic efficacy (did the animal outcome improve?) and target engagement (was a mechanism-specific biomarker modulated?). The gold-standard contrast is CAM2602, the Aurora A–TPX2 protein-protein interaction inhibitor, which showed a 19 nM binding constant, greater than 1000-fold selectivity over Aurora B, no detectable activity across a 97-kinase panel at 10 µM, and in vivo tumor arrest accompanied by biomarker modulation (increased PH3, decreased P-Thr288) (Selective Aurora A-TPX2 Interaction Inhibitors Have In Vivo Efficacy as Targeted Antimitotic Agents). The prediction is sharp: interference-flagged compounds will show phenotypic efficacy in animals at rates comparable to clean compounds, but markedly lower rates of demonstrated target-engagement biomarkers — meaning their in vivo effects are systemic stress responses, matching Herrmann et al.'s finding that broad herbal-drug cytotoxicity masks specific bioactivity in a dish and does not predict in vivo efficacy (Diversity of Pharmacological Properties in Chinese and European Medicinal Plants: Cytotoxicity, Antiviral and Antitrypanosomal Screening of 82 Herbal Drugs).
The cross-species component enters as an audit layer on the same corpus, using the BC094916 case as the cautionary template: Ruonan Xu et al. showed that murine BC094916 overexpression suppressed Creb1/Bcl2 transcription, doubled apoptosis, and cut xenograft tumor volume from 2.5 cm³ to under 1.0 cm³ — yet the human ortholog IFI16 diverges enough that the cascade is unverified in humans (BC094916 suppressed SP 2/0 xenograft tumor by down-regulating Creb1 and Bcl2 transcription). For every corpus pair we record whether in vivo engagement was shown in the same species and against the same ortholog that the in vitro assay used. We then test the compound-level and target-level factors jointly: does interference status predict lack of engagement biomarkers, and does ortholog divergence independently predict lack of cross-species engagement? The IMPROVER Species Translation Challenge supplies the honest bound here: even within one rat cell line, predicting protein phosphorylation from gene expression capped at roughly 70% balanced accuracy (AUPR about 0.4), and a minimal model using 3.4 sentinel genes per protein matched a 13,841-gene kernel pipeline (Predicting protein phosphorylation from gene expression: top methods from the IMPROVER Species Translation Challenge). So the translation gap decomposes into at least three parts — label noise (removable, our target), ortholog divergence (partially auditable), and irreducible cross-layer biological noise (ceiling-bounded at roughly 70% in the best-studied setting). Our headline deliverable is that decomposition, with error bars, on real paired data.
Baselines
The do-nothing reference point is training on raw, uncurated screening data, exactly as most public machine learning for drug discovery is done. WelQrate demonstrated that this teaches models artifacts — its control datasets contain unflagged false positives and PAINS, and sequence models trained on curated versus control data differ substantially, though no reported result found on Lacuna for the exact performance gap on the control datasets. We will reproduce this reference ourselves: same models, same splits, same tuning budget, raw labels, so every downstream claim about scrubbing has an internal number to beat.
The strongest single-mechanism competitor is the cell-count baseline of Carpenter et al. — a logistic regression on nuclei count, molecular weight, and logP that achieved an average AUC of 0.74 on the Hofmarcher benchmark, beating the CLOOME contrastive learning framework at 0.71, with the bare cell-count feature alone matching a fully connected neural network on the full Cell Painting profile (AUC 0.68 versus 0.67) (Counting cells can accurately predict small-molecule bioactivity benchmarks). It is the strongest competitor because it is the cheapest credible detector of the dominant interference mechanism, general cytotoxicity. We will run it as a scrubbing policy in isolation (drop pairs where the cell-count model predicts the activity), under the identical data, compute, and hyperparameter budget as the full multi-mechanism stack. Beating it means the full stack must recover more of the curated-versus-raw gap on WelQrate scaffold splits than the cell-count-only scrub does, and must flag in vivo non-engagement cases in the paired corpus that the cytotoxicity detector alone misses — luciferase interferers and aggregators are precisely the cases it cannot see.
The strongest overall competitor is the WelQrate wet-lab curation protocol itself — primary screen, confirmatory dose-response, counter-screen, and PAINS removal — which is the established gold standard for producing clean labels (WelQrate). Our computational scrub cannot beat wet-lab truth by definition; the fair and meaningful comparison is recoverable fraction: how much of the performance difference between models trained on wet-lab-curated and raw labels can our scrub recapture without any new experiments? We consider the proposal successful on this axis if the scrubbed models recover a majority of that difference on scaffold splits across the nine WelQrate datasets. We will also run a deliberately unfair version — scrub plus the simple descriptor-based multi-layer perceptron that Liu et al. showed matches graph neural networks — against sophisticated architectures on the same data, to confirm that label quality, not model capacity, is the operative variable.
The key ablation isolates the multi-mechanism composition: scrub with each detector alone (aggregation only, via MEGAN; optical/reporter only, via E-GuARD; cytotoxicity only, via cell count) versus the maximum-composition rule. The composition earns its complexity only if it beats every single-mechanism scrub on the paired-corpus engagement prediction, because on the in-benchmark tasks the mechanisms may be too sparse for composition to matter.
Why not just…?
Why not just apply standard PAINS substructure filters? Because the indexed evidence shows static substructure rules miss exactly the interference that matters most. Sturm et al. showed that colloidal aggregation flips on adding a single methyl group — clioquinol aggregates, methyl-clioquinol does not — which no 2D substructure matcher captures, and their MEGAN model beat the ChemAgg descriptor baseline 82% versus 73% in-distribution and 68% versus 50% on 58 external drugs (Mitigating Molecular Aggregation in Drug Discovery With Predictive Insights From Explainable AI). Tools like PrePeP extend substructure mining to PAINS characterization (PrePeP: A Tool for the Identification and Characterization of Pan Assay Interference Compounds), but the mechanism-specific ML detectors are strictly more informative for our purpose because we need a probability with a mechanism attribution, not a binary alert. Our first experiment includes the SMARTS-filter scrub as an additional single-mechanism ablation, so if it performs as well as the full stack on any task, we will report that plainly.
Why not just use WelQrate's curation and skip the computational layer? This is the most serious alternative, and we treat it as the gold standard rather than an opponent. WelQrate's limitation is scope and cost: it is confined to binary in vitro hit prediction on nine targets, requires new wet-lab triage for every dataset, and explicitly does not evaluate in vivo engagement or cross-species translation. Our proposal uses WelQrate as its calibration set and extends the curation principle computationally to the hundreds of thousands of compound-assay pairs in public corpora — including the in vitro claims underpinning published in vivo efficacy studies — where wet-lab re-triage is unaffordable. If our scrub recovers almost none of the curated-versus-raw gap, the honest conclusion would be that interference is either too subtle for current detectors or too sparse to matter; the recoverable-fraction experiment is designed to produce that answer either way.
Why not just add cell count as one more input feature to the model? Because the indexed evidence indicates that confounds must be removed, not supplied. Carpenter et al. showed that on a curated 24-target benchmark with cell-death confounders eliminated, the full Cell Painting profile reached a mean balanced accuracy of 0.67 while the cell-count baseline dropped to 0.52 — the count is a detector of the confound, not a feature carrying genuine signal (Counting cells can accurately predict small-molecule bioactivity benchmarks). A model given the confound as a feature can learn to rely on it exactly when it is most misleading (on contaminated assays). That said, feature-addition is a legitimate rival policy, so we will run it as an arm in Workstream B: same models and budgets, cell count (and detector scores) as features versus label scrubbing. If feature-addition matches scrubbing on scaffold splits, we will adopt the simpler recipe and say so.
Why not just train bigger models — graph neural networks or 3D architectures — to learn past the noise? Because two independent benchmarking efforts found that capacity is not the binding constraint. Liu et al. found a simple multi-layer perceptron on 2D descriptors routinely matched graph neural networks across all nine WelQrate datasets, with graph-model advantages evaporating under scaffold splits; Huang et al. found classical 2D descriptors performing as well as or better than advanced graph architectures across 21 methods and 24 tasks in the Therapeutics Data Commons (Therapeutics Data Commons: Machine Learning Datasets and Tasks for Drug Discovery and Development). Models cannot learn past labels that are wrong at the source; the TDC temporal-split collapse from roughly 0.7 to roughly 0.4 Pearson correlation is the symptom we are re-explaining, and we deliberately fix model class so that any improvement is attributable to the labels.
Proposed Experiments and Decisive Checks
Experiment 1 (calibration) runs the three-way comparison on WelQrate: raw, computationally scrubbed, and wet-lab-curated labels, simple descriptor MLPs, identical splits and tuning, early-enrichment metrics (logAUC, BEDROC) under Bemis-Murcko scaffold splits. No reported result found on Lacuna for per-model logAUC or BEDROC values on WelQrate, so all comparisons are internal to this experiment; the pre-registered success criterion is recovering a majority of the curated-versus-raw difference. Experiment 2 (temporal decomposition) asks whether interference prevalence in the Therapeutics Data Commons drug–target interaction data is time-stamped such that the documented 0.7-to-0.4 temporal-split collapse shrinks after scrubbing — a positive result would re-attribute part of a widely cited "distribution shift" to label noise. Experiment 3 (paired corpus) is the decisive check: on the assembled in vitro/in vivo literature corpus, interference status must predict absence of target-engagement biomarkers in vivo at rates well above chance, while phenotypic animal efficacy should be similar across flagged and unflagged compounds — the signature of stress-response pharmacology. The CAM2602 case supplies the positive control (engagement biomarkers present, interference-free by construction), and the quercetin MAFLD study supplies the critical test case, since its mechanism claims rest on a chemically promiscuous scaffold. Experiment 4 (cross-species audit) fits a joint model of engagement failure on interference flags and ortholog divergence, with the IMPROVER 70% balanced-accuracy ceiling as the explicit bound on what biology alone can deliver; the deliverable is the decomposition of the translation gap into removable label noise, auditable ortholog divergence, and irreducible cross-layer noise, each with a confidence interval.
The broader payoff is practical: if a substantial fraction of failed translation is label noise, then a free computational pre-filter — run before any target nomination or animal experiment — could save a meaningful share of the resources currently spent chasing artifacts into mice. And for target validation specifically, the corpus result would give the field a concrete answer to a question that has been begged for years: when a phenotypic screen says a target is validated, how often is the dish simply dying?
References
- Counting cells can accurately predict small-molecule bioactivity benchmarks
- WelQrate: Defining the Gold Standard in Small Molecule Drug Discovery Benchmarking
- Therapeutics Data Commons: Machine Learning Datasets and Tasks for Drug Discovery and Development
- Mitigating Molecular Aggregation in Drug Discovery With Predictive Insights From Explainable AI
- E-GuARD: expert-guided augmentation for the robust detection of compounds interfering with biological assays
- Reference compounds for characterizing cellular injury in high-content cellular morphology assays
- The mechanism study of quercetin isolated from Zanthoxylum bungeanum maxim. inhibiting ferroptosis and alleviating MAFLD through p38 MAPK/ERK signaling pathway based on lipidomics and transcriptomics
- Curcumol Induces G1 Phase Arrest in SK-Hep-1 Cells by Targeting SKP2-Mediated p27 Degradation
- Matrine induces apoptosis of human multiple myeloma cells via activation of the mitochondrial pathway
- Selective Aurora A-TPX2 Interaction Inhibitors Have In Vivo Efficacy as Targeted Antimitotic Agents
- Diversity of Pharmacological Properties in Chinese and European Medicinal Plants: Cytotoxicity, Antiviral and Antitrypanosomal Screening of 82 Herbal Drugs
- BC094916 suppressed SP 2/0 xenograft tumor by down-regulating Creb1 and Bcl2 transcription
- Predicting protein phosphorylation from gene expression: top methods from the IMPROVER Species Translation Challenge
- PrePeP: A Tool for the Identification and Characterization of Pan Assay Interference Compounds