Active Policy Optimization for Individualized Dosing via Gradient Variance Minimization

Yi Wan, Xin Wang, Huanhuan ChenICML 2026 · OpenReview

Introduction

In precision medicine and targeted marketing, finding the optimal, individualized dose is a delicate balancing act between maximizing efficacy and minimizing adverse effects. Learning these Individualized Treatment Rules (ITRs) through real-world experiments, such as clinical trials, is notoriously expensive and constrained by two major hurdles. First, strict budgets limit the number of subjects that can be tested. Second, researchers often face a "one-shot" constraint: a specific dose can only be tested on a given individual once, preventing iterative tweaking on the same subject.

Existing active learning methods in causal inference typically attempt to improve the global accuracy of a model, spending valuable budget to map out the entire dose-response curve. However, if the ultimate goal is simply to find the best dose, perfectly mapping clearly suboptimal regions is a waste of resources. This paper introduces a paradigm shift, moving away from global curve-fitting to directly targeting the certainty of the optimum.

The GVALID Framework

To address the inefficiencies of global modeling, the authors propose GVALID (Gradient Variance Active Learning for Individualized Dosing). The core logic of GVALID is built on a mathematical insight: the policy "regret"—the loss in value from failing to pick the true optimal dose—is theoretically bounded by the posterior gradient variance at the estimated optimal dose.

GVALID operationalizes this insight through several key components:

  • Gaussian Process (GP) Surrogate: The framework employs a GP to model the non-linear, continuous relationships between patient features, administered doses, and observed outcomes.
  • Gradient-Based Acquisition: Instead of selecting samples where the outcome itself is most uncertain (as in standard active learning), GVALID selects samples that maximally reduce the uncertainty of the gradient at the current predicted optimal dose. If the model is highly certain that the gradient is zero, it has likely identified the peak response.
  • Finite-Difference Proxies: Because true gradients cannot be directly observed in a one-shot experimental setting, GVALID approximates them using a symmetric finite-difference scheme.
  • Efficient Batching: Real-world trials often treat patients in cohorts. GVALID uses a greedy batch selection strategy optimized by Schur complement updates. This allows the algorithm to select a batch of informative patients and doses without the computational burden of recomputing the entire model from scratch for every candidate.

Experimental Setup

The researchers evaluated GVALID against nearly a dozen baselines—including GP-UCB, Thompson Sampling, and specialized causal inference learners like ABC3 and PG—across five distinct environments designed to stress-test the model:

  1. HNL (Hard Non-Linear): A smooth, unimodal baseline.
  2. CSC (Complex Sharp Concave): Features highly non-linear mappings and sharp response peaks.
  3. SWV (Simple Wavy): A multimodal environment that intentionally violates the unimodal assumption to test recovery from local optima.
  4. STS (Smooth Tanh Saturation): Simulates "plateau" responses common in oncology, where many doses might yield equally optimal results.
  5. News (Semi-Synthetic): A high-dimensional benchmark (501 features) based on real-world news consumer data, testing scalability in complex, sparse feature spaces.

Key Insights and Results

Stationarity Over Absolute Value

The fundamental "aha moment" of GVALID is that finding an optimum does not require knowing the absolute value of the response; it only requires knowing where the slope (gradient) is zero. Reducing gradient uncertainty proves to be a much more direct path to policy optimization than reducing overall value uncertainty.

Joint Optimization and Cold-Start Advantage

Unlike methods that select the best dose for a randomly chosen patient, GVALID jointly selects the most informative patient from a pool alongside the best dose to administer, maximizing the information gained per trial dollar. Furthermore, GVALID exhibits a strong "cold-start" advantage. While traditional bandit methods often hallucinate or chase outliers when initial data is scarce, GVALID’s focus on gradient stability keeps its search highly directed.

Superior Efficiency and Scaling

GVALID consistently outperformed competing baselines. On the HNL dataset, GVALID achieved a policy suboptimality of just 0.12 after 350 queries, significantly beating the next best active learning baseline (MaxVar at 0.18) and random sampling (0.33). In the 501-dimensional News dataset, GVALID maintained a significant lead, proving that its covariance-based mechanism effectively navigates high-dimensional spaces where standard distance-based metrics fail. Impressively, even in the multimodal SWV dataset where its core unimodal assumptions were violated, GVALID's flexible GP foundation allowed it to eventually surpass all other methods.

Limitations and Future Directions

While GVALID offers a highly efficient approach to active policy learning, it has a few notable limitations:

  • Computational Complexity: Gaussian Processes scale cubically ($O(n^3)$) with the number of samples. While GVALID utilizes efficient batch updates, scaling to massive trials may require sparse GP approximations.
  • Safety Constraints: The current framework does not explicitly incorporate a "safety cost" or "toxicity threshold" into its acquisition function. Integrating these constraints is a critical next step for Phase I clinical trials.
  • Multi-Dimensional Dosing: The present research focuses on a single continuous dose. Future iterations could extend the framework to combination therapies, navigating a multi-dimensional gradient space to optimize simultaneous drug administrations.

Summary updated

Abstract

In domains such as healthcare and marketing, learning optimal individualized dosing policies to maximize utility is crucial, yet high experimental costs impose strict budget constraints, necessitating efficient active policy learning. Existing active learning methods in causal inference primarily focus on binary treatments and effect estimation, leaving continuous dosing and policy optimization underexplored. To address this gap, we propose an active learning framework tailored for optimal policy learning. Exploiting the inherent structure of dose-response curves, we theoretically show that the policy optimization regret is bounded by the expected posterior gradient variance at the estimated optimal doses. Motivated by this result, we introduce Gradient Variance Active Learning for Individualized Dosing (GVALID), a batch acquisition strategy that greedily selects samples to minimize target gradient variance for efficient policy learning. Experiments demonstrate that GVALID achieves superior performance under strict budget constraints.
BibTeX
@inproceedings{wan2026active,
  author = {Yi Wan and Xin Wang and Huanhuan Chen},
  title = {{Active Policy Optimization for Individualized Dosing via Gradient Variance Minimization}},
  booktitle = {International Conference on Machine Learning},
  year = {2026},
  url = {https://openreview.net/forum?id=TsMlyXEEuA},
}
Figure