Letting the Dose Decide Which Model to Trust

A dose-response curve shows how a cell line or patient responds as the concentration of a drug rises. Different kinds of curve models are good at different parts of that curve. Classical four-parameter sigmoid fits are robust where data are thin, especially at extreme concentrations. Flexible neural models capture bends, plateaus and dips, but only where measurements are dense. Today one model family is chosen for the whole curve, or several are averaged with one fixed set of weights.

We propose a different combination rule. The mixing weights change continuously along the dose axis and also depend on the drug–cell pair. Each weight is learned from out-of-fold accuracy at that dose. The practical payoff is a principled referee for a long-running dispute in pharmacology: is a low-dose "bump" in a viability curve real hormesis (low doses stimulate, high doses inhibit) or plate noise? Under this scheme, a non-monotone learner gets weight at a given dose only if it predicts held-out measurements at that dose better than the monotone learners do.

The core claim is falsifiable. Model competence in dose-response prediction varies systematically with concentration. An ensemble whose weights are a function of both dose and inputs will beat the best single model and the best dose-blind ensemble on held-out curves. The gains should be concentrated at the concentration tails and in biphasic regions. It should also flag fewer spurious dips than an unconstrained flexible model while still recovering true biphasic shapes.

What exists and what is missing

Two lines of work supply the method. Bayesian hierarchical stacking makes stacking weights functions of the inputs, $w_k(x)$, with partial pooling toward global weights. It was demonstrated on polling and well-switching data, and it scores models on pointwise predictive densities. It never treats a dose axis along which the same model's competence changes. Ensemble Method for Estimating Individualized Treatment Effects stacks treatment-effect models using pseudo-outcomes. It reports beating oracle selection on 95% of ACIC datasets and cutting mean squared error by up to 48% over the best single model, but only for binary treatments. We reuse the stacking logic of both. We depart from them by indexing the weights by dose, and by working on screening data where doses are assigned by design, so no pseudo-outcome is needed.

The motivation comes from the dose-response literature itself. Predicting Dose-Response Curves with Deep Neural Networks introduces ARCANet, which uses chemical and transcriptomic encoders with a Hill-shaped prior plus residual freedom. It reports that below roughly 2,000 data points, simple 3- and 4-parameter logistic models beat it. That is direct evidence that the best model depends on how much data sits in a region.

The Bayesian tensor models of Relational Dose-Response Modeling for Cancer Drug Studies and A Bayesian model of dose-response for cancer drug studies share strength across drugs and cell lines. They enforce strict monotonicity, which suppresses screening noise but cannot represent the biphasic behavior documented for agents like curcumin (A Comprehensive Review on the Benefits and Problems of Curcumin with Respect to Human Health) and ascorbic acid (Hormetic dose response to L-ascorbic acid as an anti-cancer drug in colorectal cancer cell lines according to SVCT-2 expression). These models constrain our design: we must not force a single shape assumption.

Finally, NetBiTE is the closest ensemble in drug response. It is a random forest whose feature sampling is biased by protein-interaction networks. It predicts a scalar $IC_{50}$, the concentration that halves viability, and it discards the curve. We use it as a tree-based member of the model library rather than as a competitor on the same task.

Core Mechanism

Take a library of $K$ curve models. Each has been trained on its own cross-validation folds and outputs a predicted viability $f_k(x, s)$ for drug–cell pair features $x$ at log-concentration $s$. The ensemble prediction is

$$\hat y(x,s)=\sum_{k=1}^{K} w_k(x,s)\, f_k(x,s),\qquad w_k(x,s)=\frac{\exp\big(\alpha_k + g_k(s) + h_k(x)\big)}{\sum_j \exp\big(\alpha_j + g_j(s) + h_j(x)\big)}.$$

The terms of the weight function are defined as follows. The $\alpha_k$ are global log-weights. Each $g_k$ is a restricted cubic spline in log-dose with a handful of knots. Each $h_k$ is a small linear map from a low-dimensional embedding of $x$. Hierarchical shrinkage priors pull $g_k$ and $h_k$ toward zero, so the ensemble falls back to global stacking wherever data cannot support dose-specific preferences. This is the partial-pooling logic of hierarchical stacking, carried over to the dose axis.

The weights are fit by minimizing squared error on out-of-fold measurements at the doses actually measured. Every term is therefore scored against real data, never against a model's own interpolation.

The library is deliberately built to differ in shape assumptions. It contains a per-pair 4-parameter Hill fit; a monotone model sharing strength across pairs, either the tensor model above or a monotone neural head; a restricted cubic spline model on dose with pair-level random effects, following Restricted Cubic Splines for Non-Linear Risk Modeling; ARCANet; and a gradient-boosted tree model on concatenated drug and cell features plus dose.

The trees are included on purpose. When are ensembles really effective? finds that ensemble gains track the disagreement-error ratio (correlation $R \ge 0.96$ with ensemble improvement). Deep networks that fit training data exactly tend to make correlated mistakes, while forests do not. Mixing model families is how we keep that ratio above one.

Research Program

The work has three parts. First, we build the out-of-fold prediction tensor for every model, dataset and split, and compute the disagreement-error ratio separately in each dose decile. This gives a pre-registered prediction to test: stacking gains should appear where the ratio exceeds one and vanish elsewhere.

Second, we fit the dose-indexed stacker and its ablations, all of which are special cases of the equation above. Dose-blind stacking drops $g_k$. Input-only hierarchical stacking keeps $h_k$ but drops $g_k$. Dose-only stacking drops $h_k$. Dose-binned stacking replaces the spline $g_k$ with piecewise constants.

Third, we run the hormesis-arbitration study. Real curves are resampled and perturbed in two ways. In one, we inject true biphasic shapes of controlled amplitude and width into randomly chosen pairs. In the other, we inject structured artifacts: replicate-inconsistent dips that mimic pipette-column effects. The question is whether the ensemble shifts weight toward non-monotone members only when the dip is real.

Baselines

The naive reference point is a per-pair 4-parameter Hill fit, the standard practice in pharmacology. It is run on exactly the measured doses available in each training fold. It cannot predict unseen pairs, so for held-out pairs its reference is the Hill fit to the training-set median curve of that drug.

The strongest existing single model is ARCANet. The file reports that it outperforms parametric and machine-learning baselines on CTRPv2, GDSC, PRISM and NCI60 full-curve prediction, but gives no number: there is no reported result found on Lacuna for ARCANet's curve-level error. We will therefore retrain it on our splits with its published architecture and the same hyperparameter budget as every other member. The nearest quantified figure in the file is tCNNS's mean $R^2 = 0.826$ on GDSC (Improving prediction of phenotypic drug response on cancer cell lines using deep convolutional network). That figure is for scalar $IC_{50}$ rather than curves, so we report it only as context, not as a target.

The strongest ensemble competitor is input-dependent hierarchical stacking with weights $w_k(x)$ and no dose term. It is trained on the same out-of-fold predictions, with the same prior family and the same number of weight parameters as our dose-indexed version. Matching the parameter count means any gain comes from where the flexibility is placed, not from having more of it. Uniform averaging and dose-blind global stacking complete the ensemble baselines. The dose-binned ablation isolates whether smooth dose dependence matters beyond coarse dose-dependence.

Beating the field means three things. The dose-indexed stacker must achieve lower curve-integrated squared error than both the best single member and input-only stacking. The improvement must hold across held-out-cell, held-out-drug and held-out-concentration splits. And we require a paired bootstrap confidence interval over drug–cell pairs that excludes zero.

Why not just…?

Why not just fit 4-parameter Hill curves, as pharmacology has for decades? They are robust in sparse regimes, which is exactly what the ARCANet data-efficiency result shows. But they cannot represent biphasic shapes and cannot generalize to unmeasured pairs. The proposal does not discard them. It predicts they will receive high weight at the concentration tails, and the learned $g_k$ curves will show whether that happens.

Why not just use restricted cubic splines to capture non-monotone shapes? Splines bend where needed, but on their own they cannot use transcriptomic or chemical inputs or share strength across drugs. On noisy screens they will also happily fit plate artifacts. The injected-artifact experiment measures their false-hormesis rate directly against the ensemble's.

Why not just average the models uniformly? Averaging is the cheapest option and is often hard to beat when members are equally good. The ARCANet sparse-data finding suggests they are not equally good, and hierarchical stacking was motivated by exactly this kind of uneven competence. If uniform averaging matches our stacker on held-out curves, the central claim fails, and we will report it that way.

Why not just use a single model that already blends a Hill prior with a flexible residual, like ARCANet? This is the most serious threat. ARCANet in effect builds a parametric-plus-flexible mixture inside one network. The difference is that its blend is learned by fitting the training data, whereas ours is learned from held-out error. That matters in the regimes where the network is overconfident. ARCANet is a library member and the main single-model baseline. The held-out-concentration split, which trains on inner doses and tests on the extremes, is where we expect the distinction to show. If ARCANet alone wins there, the ensemble adds nothing.

Why not just bin doses and stack within each bin? This is a coarser version of the same idea, and it may do nearly as well. It is included as an ablation. If it ties the spline version, the finding is still that competence depends on dose; only the claim that smooth weighting matters is weakened.

Proposed Experiments

The data are CTRPv2, GDSC1/2, PRISM and NCI60, the four public screens ARCANet used. We add the organoid screen of 67 drugs by 284 organoids analyzed in the Bayesian tensor papers, which lets us test at small sample size. Splits hold out cell lines, drugs, and the top and bottom two measured concentrations; the last tests extrapolation to the tails.

Metrics are curve-integrated squared error at measured doses, error per dose decile, and replicate-consistency error, meaning prediction of held-out technical replicates. For the synthetic hormesis study we add detection area under the ROC curve for true versus artifactual dips.

Four checks are decisive. First, the dose-indexed stacker must beat input-only stacking specifically in the tail deciles. Second, the per-decile disagreement-error ratio must predict where the gains occur. Third, the learned weight on non-monotone members must rise at injected true biphasic regions and stay flat at injected artifacts. We will judge this by detection area under the curve compared against the spline and ARCANet members alone. Fourth, on the small organoid screen the stacker must shrink to near-global weights and not underperform the monotone tensor model. If the first check fails on all four large screens, the idea is wrong.