Fixing Inherited Targeting When Outreach Planners Learn From Their Own Logs

Public health programs that phone, text or send health workers to millions of mothers can reach only a small fraction of them each week. They increasingly decide whom to contact with planners learned from past call logs. Those logs were produced by an earlier targeting rule, so they share a defect that recommender-system researchers know well as popularity bias. People who were contacted often have plenty of observed responses. People who were rarely contacted have almost none. A planner fit to these logs learns most about, and keeps choosing, whoever the old rule favored. When the planner is then retrained on the logs it generated itself, the effect compounds, much as popular items in a recommender collect more clicks because they are shown more.

This note proposes to measure that compounding in budgeted health outreach and to correct it without trial-and-error experimentation on vulnerable people. The correction borrows the logic of causal popularity debiasing: train the model of how a person's engagement changes with contact history as an explicit, separate input, then plan with the contact history the new policy would produce, not the one the old policy left behind. The central evidence would come from simulated cohorts with known ground truth, of the kind a retrospective study cannot provide on its own, built at the scale and structure of published maternal-health programs.

Core Claim

Planners for restless multi-armed bandits (a model where each beneficiary is an "arm" whose engagement evolves week to week whether or not they are contacted, and a weekly budget caps how many can be contacted) are now fit to historical engagement logs. Examples are the 3-state Self-motivated/Persuadable/Lost-cause model behind Learn to Intervene (WIQL) and the history-based forecasters in CHAHAK for Kilkari. In these logs, a beneficiary's contact history plays two roles at once. It genuinely changes future engagement, through rapport or fatigue. It is also a record of whom the previous policy chose.

A planner that feeds raw contact history into its forecaster gives the old policy's choices the same standing as real responsiveness. It computes each person's priority score as if their future contact pattern will resemble their past one. We claim that this produces three things. First, estimation error in each beneficiary's priority score (the Whittle index, the per-person value of contacting them now) grows as historical exposure falls. Second, allocation share drifts toward historically over-contacted groups across retraining rounds. Third, a deconfounded planner removes most of this drift and recovers population engagement. The planner fits contact-history effects as a separate term and re-simulates that term under the candidate policy. Its advantage over inverse propensity weighting should be concentrated where the old policy almost never contacted some groups.

Concretely, for beneficiary $i$ in week $t$ with state $s$, action $a\in\{0,1\}$, fixed features $x_i$ and cumulative contact history $z_{i,t}$, we fit

$$ \text{logit}\, P(s_{t+1}=1 \mid s, a, x_i, z_{i,t}) = f_\theta(s, a, x_i) + g_\phi(s, a, z_{i,t}), $$

where $f_\theta$ captures who the person is and how they respond to contact, and $g_\phi$ captures how accumulated contact shifts that response. Identification of $g_\phi$ comes from within-person variation in $z$ over weeks. When computing indices for a candidate policy $\pi$, $z$ is rolled forward under $\pi$ rather than frozen at its logged value. This is the planning-time analogue of the do-operator on popularity in PDA: the model is told what contact history to assume instead of inheriting it from the data.

What Prior Work Does and Leaves Open

The allocation literature supplies the problem and the planners. WIQL learns Whittle indices as Q-value differences from ARMMAN-parameterized simulations. It relies on online epsilon-greedy exploration, which its own authors flag as ethically fraught. SAHELI deployed an RMAB planner that reduced engagement drops by 30.5% for over 100,000 mothers. CHAHAK showed a "history premium": a 6th-order Markov model predicts Kilkari behavior 15% better than a 1st-order one. Its MA-TARI ranking gave a 57% increase in content consumed over random allocation. GROUPS clusters similar beneficiaries to scale optimization. DLM has an LLM write reward functions that steer allocation toward named subgroups, reaching a mean normalized reward of 0.93 versus 0.57 for a default policy, but only in simulation.

These papers motivate the proposal. CHAHAK's history premium means contact history is exactly the input that matters most, and none of these papers asks whether that input encodes the old policy rather than the person. GROUPS-style pooling is a likely amplifier of the bias. Pooled parameters are fit mostly to data-rich, heavily contacted clusters, the same gradient-volume argument that Adap-τ makes for embedding norms of frequent items. DLM's subgroup steering operates on transition models that may already be biased by past contact.

Engagement Scoring for Care-gap Intervention supplies the key conceptual distinction: high risk is not the same as high responsiveness. Its impactability ranking improved outreach efficiency by 288% for the top 100 members. However, it is a static supervised score trained on claims that were themselves shaped by past outreach.

The popularity-bias literature supplies the method. MACR subtracts a learned direct popularity effect at test time. PDA deconfounds popularity during training and then deliberately injects a chosen popularity level at inference, because some popularity reflects real quality; it reports gains of up to 119% on Kwai. PDA's two-step logic is the one we reuse. PDA, however, ranks top-K items for one user with no shared budget, and its "intervention" is passive exposure that does not change a person's future state. Our setting has a global weekly budget and actions that change future engagement. That is why the right planning-time intervention is to roll contact history forward under the candidate policy, not to set it to a constant. Taming Recommendation Bias with Causal Intervention on Evolving Personal Popularity supports treating exposure as time-varying and per-person rather than as a static count.

The offline clinical reinforcement learning work constrains the design. DeepVent uses conservative Q-learning, which penalizes actions rarely seen in the data, and OGSRL adds an out-of-distribution guard. Both anchor policies to historical practice, which is appropriate when rare actions may be unsafe. In outreach, a phone call to an under-contacted mother is not dangerous. Her rarity in the logs reflects budget and habit, not risk. Importing conservatism wholesale would therefore entrench the bias we want to remove.

The retrospective-cohort literature frames the identification problem. Tewari et al. show confounding by indication, where treatment follows patient condition, and its standard remedy, propensity matching. They also concede that unmeasured confounders survive matching.

Research Program

The work has three parts.

Diagnosis. We build semi-synthetic cohorts in which the true transition dynamics are known. Each beneficiary's response depends on features, on state history up to six weeks back (matching CHAHAK's history premium), and on a contact-history term whose sign and size we vary to represent rapport or fatigue. Each cohort is then logged under a realistic historical policy. One is access-driven targeting, where health workers favor households correlated with certain features. The other is a prior Whittle planner with a misspecified model, in the spirit of SAHELI. A temperature parameter controls how concentrated the old policy's contacts are, and therefore how much overlap the logs have.

On these logs we fit naive planners and track two things across up to five retrain-on-own-logs rounds: index error stratified by historical exposure decile, and allocation share by stratum. The main signature of the bias is a slope of index error against exposure, together with growing concentration of contacts across rounds, measured by the Gini coefficient of contacts across feature strata.

Correction. We implement the separable estimator above with $f_\theta$ and $g_\phi$ as small networks over the history window. Whittle indices are computed by simulating forward under each candidate policy with $z$ updated by that policy's contacts. Following PDA's observation that popularity can carry real signal, $f_\theta$ keeps everything the old policy's targeting revealed about responsiveness through the features. Only the contact-history pathway is re-simulated.

Generalization check. We take ARMMAN-like dynamics (the 3-state model at the budgets WIQL used) and Kilkari-like history-dependent dynamics as the two simulator families. Where a partner deployment log with a randomized comparison slice is available under a data agreement, as in the field trial reported by Torkamani et al., we use that slice as held-out ground truth for policy value. If no such log is available, we say plainly that real-data claims are limited to fit diagnostics.

Baselines

The do-nothing reference is uniform random allocation at the same weekly budget. It is the comparison CHAHAK used for its 57% figure, and it bounds how much any learned planner helps. An oracle Whittle planner with the true simulator dynamics defines zero regret, and all methods are reported as a fraction of the gap between random and oracle.

The strongest existing method is a history-aware Whittle planner fit naively to logs: a forecaster over the six-week history window, including raw contact history, feeding index computation, as in CHAHAK's MA-TARI and SAHELI. Reported results are a 57% increase in content consumed over random (CHAHAK) and a 30.5% reduction in engagement drops (SAHELI). Both are deployment or simulation figures on their own data, not on our semi-synthetic benchmark. For WIQL, the file reports only that it outperformed Myopic and Greedy at budgets M=500, 1000 and 1500 for N=5000; no reported single number for WIQL was found on Lacuna. We run the naive planner and every competitor with the same forecaster architecture, the same logs, the same number of hyperparameter trials on a held-out log split, and the same compute. Under this setup, the only difference between the naive planner and ours is how contact history is handled.

The strongest debiasing competitor is doubly robust estimation of transitions: each logged transition is weighted by the inverse of the old policy's contact probability, with an outcome-model correction, and propensities are clipped at a tuned threshold. It shares our tuning budget.

A pooled GROUPS-style planner tests whether clustering helps or amplifies the bias. A static impactability ranker in the Torkamani style, refit each week, tests whether sequential planning matters at all.

Two ablations isolate the key ingredient. The first trains the separable model but freezes $z$ at its logged value during planning. The second drops $z$ entirely. If the first ablation matches the full method, the gain comes from the split during training, not from re-simulating contact history.

We count the proposal as beating the naive planner if it closes at least a third of the remaining regret gap in the low-overlap regime across both simulator families, with confidence intervals over 20 seeds that exclude zero. We count it as beating doubly robust estimation if it wins at low overlap while staying within noise at high overlap.

Why not just…?

Why not just use inverse propensity or doubly robust weighting? This is the real threat. When the old policy contacted everyone with reasonable probability, reweighting corrects selection on observed state and should match our estimator. We expect it to fall short in two places. In low-overlap logs, weights explode; MACR's authors report that propensity weighting is hard to tune in exactly this long-tail regime. And reweighting fixes which transitions are counted but still feeds the logged contact history into the forecaster, so the planner keeps inheriting the old targeting. The overlap sweep settles this. If doubly robust estimation matches us at every temperature, the contribution shrinks to the diagnosis, and we will report it that way.

Why not just randomize a small slice of the budget? A held-out random slice is the cleanest fix, and programs may be willing to run one. It costs contacts that could have gone to likely responders, which is the ethical tension WIQL raises. It also takes weeks to accumulate data for rare groups. We measure this directly by asking what randomized fraction (1%, 5% or 10% of the budget, over the same horizon) the naive planner needs to match the deconfounded planner run with no randomization. If 1% suffices, programs should simply randomize, and we will say so.

Why not just use risk scores with propensity matching? Ranking matched cohorts by predicted risk ignores responsiveness. Torkamani et al. found risk-style heuristics far behind impactability, and WIQL found myopic and greedy rules lose to index policies over 80 weeks. The static-ranker baseline above checks that this still holds in our simulators.

Why not just add an optimism bonus for under-observed beneficiaries? Optimism does correct under-exploration. But it is online exploration in disguise and spends budget on uncertainty rather than on expected benefit. It also does not separate the person from their contact history. We include it as a secondary baseline at the same tuning budget.

Proposed Experiments and Decisive Checks

We first confirm our PDA implementation on Kwai against its reported gains of up to 119%, so that any failure in health settings cannot be blamed on a bad port. The main grid crosses two simulator families, three budgets (M=500, 1000 and 1500 of N=5000, following WIQL), four old-policy temperatures, and two signs of the contact-history effect, with 20 seeds each. Metrics are fraction of beneficiary-weeks engaged as a share of the random-to-oracle gap, index error by exposure decile, and contact Gini by feature stratum after each retraining round.

There are three decisive checks. First, if naive planners show no slope of index error against exposure and no drift across rounds, the premise is wrong and the correction is unnecessary. Second, if doubly robust estimation matches us at low overlap, the method adds nothing beyond reweighting. Third, if the frozen-$z$ ablation matches the full method, the planning-time re-simulation, which is the main departure from PDA, is not what does the work.