In drug discovery, the same molecule-target pair is often measured several times — once in an academic lab, once in a high-throughput industrial screen, once more in a published paper — and the numbers disagree. Current machine learning practice handles this badly. Multi-task learning, where one network is trained on many prediction tasks at once, is the standard answer to label scarcity in molecular property prediction: with only hundreds of annotated compounds per assay, a model that also learns from hundreds of neighboring assays does better than any single-task model. But these models treat every label as equally true. A carefully replicated measurement and a single noisy screening hit enter the loss with the same weight, and conflicting duplicates are either averaged blindly or dropped arbitrarily.
This proposal applies an old and rigorous statistical discipline to fix that. Meta-analysis — the method clinical medicine uses to pool results from independent, partly contradictory studies — assigns each study a weight proportional to its precision and explicitly estimates how much the studies genuinely disagree. We propose to run this machinery inside the training loop of a multi-task molecular model: pool conflicting cross-source measurements into consensus labels with quantified uncertainty, weight both labels and tasks by that uncertainty using a stabilized deep inverse-variance scheme, and let a population of risk-diverse search agents learn which assays should share representations, replacing the fixed protein-interaction graph that prior work depends on.
Core Claim
The claim is concrete: on sparse multi-task molecular benchmarks, most of the gap between current multi-task models and their ceiling is not missing architecture capacity but mishandled label reliability. Structured Multi-task Learning for Molecular Property Prediction showed that adding task structure to multi-task molecular learning lifts ROC-AUC on the sparsest ChEMBL subset from 74.83% (standard multi-task learning) to 78.35%; we claim that a further, comparable gain is available from the labels themselves — by pooling replicate measurements meta-analytically and weighting the loss by their precision — and that this gain is largely independent of the backbone, applying to graph networks and to classical cheminformatics descriptors alike.
What Exists and What Is Missing
Three lines of work bound the idea. Multi-task molecular learning is best represented by Structured Multi-task Learning for Molecular Property Prediction, whose SGNN-EBM model passes messages across a task graph built from known protein-protein interactions and captures label dependencies with an energy-based output head; it leaves open that its task graph is fixed a priori and that it has no mechanism to arbitrate conflicting assay observations or estimate assay-level noise. The meta-analytic side is supplied by Batch Inverse-Variance Weighting: Deep Heteroscedastic Regression, which shows that the textbook rule of weighting each data point by the inverse of its noise variance makes deep networks diverge (near-zero variance estimates produce infinite gradients), and fixes this by normalizing weights within each mini-batch and adding a smoothing floor; but it was evaluated only on single-task image and tabular data with synthetic noise, never on multi-task architectures or molecular data. On the agent side, LLM-based systems such as MADD and CLADD coordinate heterogeneous agents over drug-discovery queries, and Manalyzer automates full meta-analysis pipelines over scientific literature — but the former resolve conflicting evidence with qualitative prompting and the latter only mines text; neither trains a predictive model. No existing system combines formal statistical pooling with multi-task gradient learning on molecular data, and none weights molecular labels by measurement precision. That is the gap this proposal occupies.
Clinical meta-analysis supplies both the method and the precedent that pooling resolves apparent contradiction: Has the Impact of Helicobacter pylori Therapy on Ulcer Recurrence in the United States Been Overstated? shows pooling resolving inconclusive individual studies, and Correlation of vitamin D receptor gene (ApaI) polymorphism with periodontitis: A meta‐analysis of Chinese population shows that decomposing between-study disagreement ($I^2$, $\tau^2$) reveals why studies conflict — regional subpopulations — rather than just that they do. Our claim is that conflicting assay readouts for the same compound decompose the same way: protocol and source differences, not measurement chaos.
Research Program
Three components are built, in order of dependency.
Evidence pooling layer. For every compound-assay pair measured in more than one source, we run a random-effects meta-analysis — pooling that explicitly estimates between-source disagreement — to produce a consensus label $\hat{y}_i$ and an uncertainty $\hat{\sigma}_i$. Under a standard random-effects model, the variance of the pooled estimate for compound $i$ in assay $t$ is
$$\hat{\sigma}_i^2 \;=\; \frac{\hat{\tau}_t^2}{k_i} \;+\; \overline{s}_i^{\,2},$$
where $k_i$ is the number of independent source measurements, $\overline{s}_i^{\,2}$ their average within-source variance, and $\hat{\tau}_t^2$ the between-source heterogeneity of assay $t$. Single-measurement compounds keep their raw label with an assay-level variance estimate. The per-assay heterogeneity $\hat{\tau}_t^2$ is itself a product: it becomes the assay's precision score, separating trustworthy assays from noisy ones.
Heteroscedastic multi-task objective. The pooled labels feed a loss that extends Batch Inverse-Variance Weighting from single-task to multi-task learning:
$$\mathcal{L} \;=\; \sum_{t} \rho_t \, \frac{1}{|\mathcal{B}_t|}\sum_{i \in \mathcal{B}_t} \bar{w}_i \,\big(f_t(x_i) - \hat{y}_i\big)^2, \qquad \bar{w}_i = \frac{(\hat{\sigma}_i^2 + \epsilon)^{-1}}{\sum_{i' \in \mathcal{B}_t} (\hat{\sigma}_{i'}^2 + \epsilon)^{-1}},$$
where $f_t$ is the assay-$t$ prediction head, $\mathcal{B}_t$ the assay's samples in the mini-batch, $\epsilon$ the variance floor that prevents divergence, and $\rho_t = (\hat{\tau}_t^2 + \overline{s}_t^{\,2})^{-1}$ the assay-level precision that scales each task's contribution. The within-batch normalization is what makes this trainable where naive inverse-variance weighting explodes; the task-level term is the new ingredient, and it directly addresses the negative-transfer problem documented by RotoGrad: Gradient Homogenization in Multi-Task Learning, where tasks with dominant gradients hijack the shared representation — except we weight by evidential reliability rather than by convergence speed.
Risk-diverse discovery of the task graph. SGNN-EBM's task graph comes from a fixed protein-interaction database and, as its authors acknowledge, fails when those relations are incomplete. We replace it with a search process structured as heterogeneous agents, importing the mechanism of Behaviorally Heterogeneous Multi-Agent Exploration Using Distributed Task Allocation: each search agent carries a risk parameter $\alpha$ (a single number describing how cautiously or boldly it treats uncertain outcomes). Cautious agents ($\alpha < 1$) only propose representation-sharing edges between assays backed by strong protein-interaction evidence and low measured heterogeneity; bold agents ($\alpha > 1$) propose edges into sparse, unrelated target families where no biology is documented. A best-response allocation over validation gain selects the final edge set. The prior finding that behavioral heterogeneity acts as a tie-breaker preventing redundant exploration motivates the prediction that mixed $\alpha$ teams will find better graphs than homogeneous ones. This is the most speculative component, and it is scoped as a second research question, not a load-bearing assumption: the pooling and weighting layers are valuable even on a fixed graph.
Proposed Experiments
A feasibility audit comes first: quantify what fraction of compound-assay pairs in merged ChEMBL and BindingDB data have two or more independent source measurements, and report per-assay $I^2$. If replicate coverage is substantial, the full pooling pipeline is justified; if it is thin, the method degrades gracefully to assay-level noise weighting, and we will say so plainly — that outcome would itself be a useful negative result about what "conflicting evidence" in molecular databases actually consists of.
The main comparison runs on the ChEMBL 10, 50, and 100 multi-task benchmarks from Structured Multi-task Learning for Molecular Property Prediction, using its evaluation protocol, with ROC-AUC as the primary metric. That paper reports 78.35% for SGNN-EBM, 74.83% for standard multi-task learning, and 71.67% for single-task learning on ChEMBL 10; no reported result found on Lacuna for ChEMBL 50 or ChEMBL 100, so those subsets require rerunning the baselines ourselves under matched conditions.
A robustness test injects controlled conflicts: duplicate a fraction of labels with divergent values and variances, and measure how each method degrades. Equal-weight multi-task learning should degrade linearly in injected noise; the pooled model should absorb it, with degradation governed by the estimated $\hat{\sigma}$ rather than the raw discrepancy.
Out-of-distribution evaluation follows the protocol of Therapeutics Data Commons: Machine Learning Datasets and Tasks for Drug Discovery and Development, under scaffold and temporal splits. This is deliberate self-exposure: TDC's own benchmarking found that classical 2D descriptors (RDKit2D) with simple models often match or beat deep graph networks under such splits, with drug-target interaction Pearson correlation dropping from roughly 0.7 in-distribution to roughly 0.4 out-of-distribution. We therefore apply the meta-analytic weighting to two backbones — a graph network and a descriptor-based model — to test the claim that the gain comes from label quality, not architecture.
Finally, graph ablations compare the fixed protein-interaction graph, the agent-discovered graph, and no graph, including an edge-deletion test that removes protein-interaction entries to simulate incomplete biology; and an agent-population ablation compares mixed-$\alpha$ teams against homogeneous cautious and homogeneous bold teams. For the DARPA SubT exploration result that motivates mixed teams, no reported result found on Lacuna for exploration times, so the transfer of that finding to task-space search is a hypothesis this ablation tests rather than an established fact.
Baselines
The naive reference point is single-task learning: one model per assay, trained independently. On ChEMBL 10 this reaches 71.67% ROC-AUC as reported by Structured Multi-task Learning for Molecular Property Prediction; it is included because it is what a practitioner does today without any multi-task machinery, and because TDC's evidence means its classical-descriptor variant is stronger than it looks. It will be run with both a graph backbone and a random forest over RDKit2D descriptors, under the same splits and tuning budget as every other method.
The intermediate baseline is standard multi-task learning with equal loss weighting — one shared backbone, losses summed across assays. It is the strongest cheap competitor because it already captures most of the benefit of cross-assay sharing; it reports 74.83% ROC-AUC on ChEMBL 10 in the same paper. It will be trained on both the raw conflicting labels and, in an ablation, on our pooled consensus labels with equal weights, which isolates the value of pooling alone from the value of weighting.
The strongest existing method is SGNN-EBM from Structured Multi-task Learning for Molecular Property Prediction, which combines the protein-interaction task graph with an energy-based output structure and reports 78.35% ROC-AUC on ChEMBL 10. To make the comparison fair, we rerun it on identical ChEMBL subsets and splits, with a backbone of matched parameter count and an equal hyperparameter search budget, reporting both our reproduction and its published number. Beating it means exceeding 78.35% ROC-AUC on ChEMBL 10 under its own protocol — and, more informatively, exceeding it in the ablation that keeps its fixed graph and adds only our meta-analytic weighting, which would show the new ingredient rather than confounded architecture changes carries the gain.
The key ablations are: pooled labels with equal weights (isolates pooling), inverse-variance weights with raw labels (isolates weighting), and the full system with a learned versus fixed graph (isolates the agent component). If the weighting-only ablation already matches SGNN-EBM, the core claim is confirmed even if the full system's graph discovery adds little.
Why not just…?
Why not just train single-task models on classical descriptors? This is the most serious objection, because Therapeutics Data Commons found that descriptor-based simple models often match deep graph networks exactly where we intend to evaluate. We do not argue around it: we run that baseline, and we apply our weighting to it. If meta-analytic labeling improves the random forest too — which the core claim predicts, since better labels should help any learner — then the objection dissolves into an agreement that label quality is the binding constraint. If it helps only graph networks, we will report that and revise the claim.
Why not just sum the task losses (standard multi-task learning)? Equal weighting lets a noisy high-throughput screening hit shape the shared representation as strongly as a replicated measurement, and the negative-transfer consequence is documented: RotoGrad shows fast-converging tasks dominate shared gradients, and on ChEMBL 10 equal weighting trails structured multi-task learning by 3.5 ROC-AUC points. The counter-experiment is the conflict-injection test: if equal weighting degrades no faster than precision weighting, the mechanism is wrong.
Why not just apply textbook inverse-variance weighting, or filter out noisy points? Batch Inverse-Variance Weighting answers this directly: naive inverse-variance weighting diverges in deep networks, returning no result across its evaluations, because near-zero variance estimates produce exploding gradients; hard noise cutoffs discard real signal in moderately noisy samples and lost to the stabilized scheme on every noise distribution tested. We reuse the stabilized version rather than reinventing it — the open question it leaves is whether it works in multi-task molecular settings, which is precisely what we test.
Why not just prompt one large LLM, or use the existing multi-agent chemistry systems? Monolithic prompting fails hard on this domain: chain-of-thought prompting achieved a 0% valid code rate on drug-target interaction tasks where the multi-agent DrugAgent reached 87.5–100%, and single-model GPT-4o scored F1 63.2 on screening where Manalyzer reached 76.8. More fundamentally, MADD and CLADD resolve conflicting sources with qualitative prompting and train no shared representation, so they cannot produce the deliverable here — a predictive model scored by ROC-AUC on held-out compounds. Honest caveat: if the feasibility audit shows the molecular databases themselves lack numeric replication, a Manalyzer-style literature-extraction front end becomes necessary to supply the measurements, and we would treat that as complementary rather than competing.
Why not just pre-train a bigger molecular model? GROVER showed a 23.9% relative improvement on the small FreeSolv dataset from self-supervised pre-training on 10 million molecules, but TDC's out-of-distribution results show pre-training does not fix scaffold or temporal shift, and pre-training is orthogonal to our question: it adds capacity learned from unlabeled molecules, while leaving conflicting labels unresolved. The two compose, and our weighting applies unchanged to a pre-trained backbone.
Decisive Checks
Three results settle the proposal. First, the weighting-only ablation exceeding 78.35% ROC-AUC on ChEMBL 10 with SGNN-EBM's own protocol, ideally reproduced across both backbones, would confirm that label reliability — not architecture — is the binding constraint in sparse multi-task molecular learning. Second, the conflict-injection curve, showing precision-weighted degradation governed by estimated uncertainty while equal weighting tracks raw noise, would confirm the mechanism. Third, the feasibility audit's replicate-coverage number decides which version of the claim is true in practice: broad replication makes the full pooling story correct; thin replication means the honest contribution is assay-level precision weighting, and the field's "conflicting evidence" problem lives more in the literature than in the databases — a finding worth publishing either way.
References
- Structured Multi-task Learning for Molecular Property Prediction
- Batch Inverse-Variance Weighting: Deep Heteroscedastic Regression
- MADD: Multi-Agent Drug Discovery Orchestra
- RAG-Enhanced Collaborative LLM Agents for Drug Discovery
- Manalyzer: End-to-end Automated Meta-analysis with Multi-agent System
- Has the Impact of Helicobacter pylori Therapy on Ulcer Recurrence in the United States Been Overstated?
- Correlation of vitamin D receptor gene (ApaI) polymorphism with periodontitis: A meta‐analysis of Chinese population
- RotoGrad: Gradient Homogenization in Multi-Task Learning
- Behaviorally Heterogeneous Multi-Agent Exploration Using Distributed Task Allocation
- Therapeutics Data Commons: Machine Learning Datasets and Tasks for Drug Discovery and Development
- DrugAgent: Automating AI-aided Drug Discovery Programming through LLM Multi-Agent Collaboration
- GROVER: Self-supervised Message Passing Transformer on Large-scale Molecular Data