When one graph neural network is trained to predict many properties of a molecule at once, every task sees the same neighborhood of each atom. A graph neural network here means a model that updates each atom's representation by pooling information from atoms it is bonded to. That shared view is a hidden constraint. One assay may respond to a reactive group spanning two bonds, while another depends on how polar groups are arranged across the whole molecule. We propose to make "how far each task looks" an explicit, measurable part of the shared model. The backbone is built so that each bond distance $k$ has its own parameter block. We then measure, block by block, whether different tasks push those parameters in opposite directions. Where tasks agree, the block is shared. Where they persistently disagree, the block is copied for separate groups of tasks.
The bet is that negative transfer in multi-task graph models is not spread evenly through the network. Negative transfer is the familiar case where training on several tasks together makes some of them worse than training alone. We expect it to concentrate at particular hop distances, and expect that splitting only those distances recovers most of the benefit of separate models while keeping most of the benefit of sharing. If it does not concentrate this way, the per-hop measurements will show that directly, and that is a useful negative result for anyone designing multi-task graph models.
Core Claim and What Is New
Multi-hop aggregation methods let a single layer read atoms at exact distance $k$ directly, instead of reaching them only by stacking $k$ layers. Power up! does this with weighted sums of exact-distance adjacency matrices $A^{[k]}$. Tree Decomposed Graph Neural Network builds separate per-hop matrices and learns how to weight them. Multi-hop Attention Graph Neural Network spreads attention across paths with a PageRank-like decay. All of these are evaluated on one task at a time.
The closest existing work, Adaptive Distance Message Passing From the Multi-Relational Edge View, learns a separate hop reach for each bond type. It is evaluated only on single-task ZINC and MolHIV, and it conditions reach on the type of edge, not on the task being predicted.
On the multi-task side, the existing work models relationships between tasks but not between tasks and distance. Structured Multi-task Learning for Molecular Property Prediction runs message passing over a graph of tasks derived from protein interactions, and adds an energy model over joint labels. It still extracts the same structural features from every molecule for every task. MoleculeNet shows that joint training of graph models acts as a regularizer on Tox21, but its models share one layer-stacked receptive field.
What is new here is the unit of analysis. We localize task conflict to hop-specific parameters and use that localization to decide where the network branches. Neither line of work does this.
Research Program
The backbone is a hop-factored message-passing layer. Each atom $v$ is updated as $h_v' = \sigma\big(\sum_{k=1}^{K} W_k \sum_{u: A^{[k]}_{vu}=1} h_u\big)$. Here $W_k$ is the weight matrix applied to atoms at exact shortest-path distance $k$, $K$ is the maximum distance (we use 6), and $\sigma$ is a nonlinearity. On drug-sized molecules, all-pairs shortest paths are cheap, so the quadratic cost flagged in How Powerful are K-hop Message Passing Graph Neural Networks is not binding.
During a warm-up phase of ordinary joint training, we record a conflict score for every hop block $k$ and every pair of tasks $(i,j)$:
$$c_k(i,j) = \mathbb{E}_{\text{batches}}\left[\frac{g_i^{(k)} \cdot g_j^{(k)}}{\|g_i^{(k)}\|\,\|g_j^{(k)}\|}\right],$$
where $g_i^{(k)}$ is the gradient of task $i$'s loss with respect to the parameters of $W_k$ alone. Negative averages mean the tasks persistently pull that distance block in opposite directions. This is the conflict condition that Gradient Surgery for Multi-Task Learning identifies. The difference is that we measure it per block rather than over the whole parameter vector.
For each $k$, we cluster tasks on the matrix $c_k$. The block is duplicated only into as many copies as there are clusters with negative mean cross-cluster conflict, and each task group uses its own copy. To keep this practical at 400 tasks, clusters are capped at a small number per hop, and the split is made once, after warm-up.
The analysis has two parts. First, we ask whether conflict is concentrated in a few hop blocks or spread uniformly. Second, we ask whether the task groupings found at different distances differ from each other. If they do, task relationships really are distance-dependent, and a single task-relation graph like the one in SGNN-EBM cannot represent them.
We validate the measurement on a synthetic suite before trusting it on real assays. Real ZINC molecules are relabeled with tasks defined by the presence of motif pairs at planted bond distances, for example two hops for some tasks and five for others. This gives ground truth for which hops should conflict.
Baselines
The naive reference point is single-task learning: one hop-factored model per task with the same architecture and per-task tuning. On ChEMBL 10, Structured Multi-task Learning for Molecular Property Prediction reports single-task ROC-AUC of 71.67% against 74.83% for standard multi-task learning. That gap is what any splitting scheme must preserve.
The plain shared hop-factored model with one output head per task is the second reference. It isolates the effect of splitting.
The strongest existing method on the main benchmark is SGNN-EBM, which reports 78.35% ROC-AUC on ChEMBL 10 in the same paper. It is strongest because it already exploits task structure, which is exactly what our method claims to do better in a distance-aware way. We will run it with its released task graph. Because its gains come from the task-graph and label-energy components rather than the molecular encoder, we will also test our hop-split encoder inside SGNN-EBM. Beating it means a higher mean ROC-AUC over the same splits and seeds. A gain that only appears in combination would still show that the two contributions are complementary.
PCGrad and RotoGrad (RotoGrad: Gradient Homogenization in Multi-Task Learning) are applied to the shared hop-factored model. Edge-type adaptive hops, following ADMPE, are applied as a shared encoder. A shared deep stack of ordinary one-hop layers with matched receptive field and parameter count completes the set.
The key ablation is random splitting: the same number of duplicated blocks, placed at randomly chosen hops and with random task groups. This isolates whether the conflict measurement, rather than the added capacity, does the work.
All methods get the same data splits, an equal parameter budget (shared models are widened to match the split model's size), and the same number of hyperparameter trials. Results are reported over five seeds.
Why not just…?
Why not just give each task head its own learned mix over hop features? This is the most serious threat. Concatenating the per-hop representations from the last layer and letting each linear head weight them costs a few parameters per task. It already makes the receptive field task-specific. If conflict lives only in which distances matter at readout, this baseline will match our method. Our claim is narrower: the shared $W_k$ transformations in earlier layers are where tasks fight. We include per-task readout mixing as a baseline, and also add it on top of our model. If it closes the gap on both the synthetic suite and ChEMBL, we will report that splitting is unnecessary and that per-task readout is the method to use.
Why not just apply PCGrad or RotoGrad to a shared model? These methods fix gradient directions during optimization, but every task still has to live with one compromise value of each block. RotoGrad itself argues that editing gradients alone does not remove the underlying disagreement in the features. The per-hop conflict scores tell us whether the conflict is persistent, which projection cannot resolve, or merely noisy, which projection handles well. If PCGrad on the shared model matches hop splitting, the conflict was the second kind.
Why not just train separate models? The ChEMBL 10 numbers above show single-task learning losing about three ROC-AUC points to plain sharing. MoleculeNet reports that sharing matched baseline accuracy on Tox21 with one third of the training data. Separate models throw that away. We will also report accuracy for the most data-scarce tasks, where this cost should be largest.
Proposed Experiments
The main experiments run on ChEMBL 10, using its reported ROC-AUC figures for comparison, and on Tox21's twelve assays, where we found no reported numeric result on Lacuna for the graph models. These are preceded by the planted-distance synthetic suite.
The first decisive check is synthetic. The conflict matrices must recover the planted hop distances, and hop splitting must approach single-task accuracy on the conflicting tasks while beating it on tasks with little data. If the measurement cannot find planted conflict, the method should not be trusted on real assays.
The second check is on ChEMBL and Tox21. We report how concentrated conflict is across hops, using the share of total negative conflict mass held by the top two blocks. We also report how much the task groupings at different hops overlap, using adjusted Rand index between the per-hop clusterings.
The third check is whether hop splitting beats both random splitting and per-task readout mixing at matched size. If both conditions hold, the claim stands. If neither holds, we will have shown that task-dependent receptive fields are not where multi-task graph models lose accuracy, which is itself worth knowing before anyone builds more elaborate task-adaptive aggregation.
References
- Power up! Robust Graph Convolutional Network via Graph Powering
- Tree Decomposed Graph Neural Network
- Multi-hop Attention Graph Neural Network
- Adaptive Distance Message Passing From the Multi-Relational Edge View
- Structured Multi-task Learning for Molecular Property Prediction
- MoleculeNet: a benchmark for molecular machine learning
- How Powerful are K-hop Message Passing Graph Neural Networks
- Gradient Surgery for Multi-Task Learning
- RotoGrad: Gradient Homogenization in Multi-Task Learning