One Training Run for Every Confounding Budget

Causal conclusions drawn from observational data, such as "this treatment policy beats the status quo" or "this sentencing model is fair," usually rest on the assumption that no hidden variable drives both the treatment and the outcome. Sensitivity analysis drops that assumption. It introduces a number $\Gamma \ge 1$, the sensitivity parameter, that caps how far hidden factors can push someone's odds of treatment. The analysis then reports the best and worst case consistent with that cap. Nobody can measure the right $\Gamma$ from the data, so the honest output is a curve: the conclusion as a function of $\Gamma$, plus the breakdown value where the conclusion flips. When the model itself must be trained to respect the worst case, for example a predictor constrained to be fair under any confounding up to $\Gamma$, each point on that curve costs a full constrained training run. This is the same cost structure that makes adversarial training expensive: a hidden "adversary" (here, nature choosing worst-case confounding) must be re-solved for every setting of the budget.

We propose to train a single network conditioned on $\Gamma$ that matches, at every budget, the model one would get by training separately at that budget. The approach borrows two ideas from cheap adversarial training. The first is to replace the inner worst-case search with a closed form wherever one exists. The second is to share weights across perturbation levels. Our new ingredient is a small multiplier network $\lambda_\phi(\Gamma)$, a learned function that sets how strongly the fairness or robustness constraint is enforced at each $\Gamma$. With it, one augmented-Lagrangian run handles a whole continuum of constraint levels. An augmented-Lagrangian run is the penalty-plus-dual-variable method used to enforce constraints during training. The result is the full sensitivity frontier and the breakdown $\Gamma$ at roughly the cost of a few training runs, not dozens.

Core Claim

The claim is narrow on purpose. Evaluating a fixed model across $\Gamma$ is already cheap: under the marginal sensitivity model, Efficient and Sharp Off-Policy Learning under Unobserved Confounding gives closed-form sharp bounds that depend on $\Gamma$ only through a conditional quantile at level $\tau = \Gamma/(1+\Gamma)$. So once the full conditional quantile function is estimated, bounds for every $\Gamma$ follow at no extra cost. The expensive case is learning, where the optimal predictor or policy itself changes with $\Gamma$. That is the case in Causal Fairness under Unobserved Confounding: A Neural Sensitivity Framework. It trains a predictor under a worst-case counterfactual-unfairness constraint with an augmented Lagrangian, and its authors concede that this adds cost over ordinary training. It also fixes $\Gamma$ in advance, so a fairness–accuracy frontier needs a fresh run per budget. The same holds for policy learning under the sharp bound.

We claim that a $\Gamma$-conditioned model trained on randomly sampled budgets, with a learned multiplier function, recovers the per-$\Gamma$ independently trained solutions to within seed-to-seed noise. We claim it does so at a total cost of at most about three single runs. We also claim the multiplier function is necessary: a single shared multiplier will under-enforce the constraint at large $\Gamma$ and over-enforce it at small $\Gamma$.

Research Program

For the fairness setting, write $f_\theta(x,\Gamma)$ for the predictor, which takes the budget as an input. Write $\ell(\theta,\Gamma)$ for its task loss and $U_\Gamma(\theta)$ for the worst-case unfairness bound at budget $\Gamma$, computed as in the neural sensitivity framework. Let $c$ be the allowed unfairness and $p(\Gamma)$ a sampling distribution over, say, $[1,3]$. We solve

$$\min_\theta \max_\phi \; \mathbb{E}_{\Gamma\sim p}\Big[\ell(\theta,\Gamma) + \lambda_\phi(\Gamma)\,g_\Gamma(\theta) + \tfrac{\rho}{2}\,g_\Gamma(\theta)_+^2\Big], \qquad g_\Gamma(\theta)=U_\Gamma(\theta)-c,$$

where $\lambda_\phi(\Gamma)\ge 0$ is a small monotone network and $\rho$ is the penalty weight. Each minibatch draws fresh values of $\Gamma$. The multiplier network takes dual-ascent steps on the constraint violation measured at those values. This replaces the scalar multiplier of a fixed-$\Gamma$ run with a function that is shared and smoothed across budgets. We condition on $\Gamma$ through feature-wise scaling of hidden layers. This plays the same role as the perturbation embedding fed to the hypernetwork in Parameter-Saving Adversarial Training, which generates specialist weights for $\ell_1/\ell_2/\ell_\infty$ attacks. Unlike that method, we need one forward pass at test time, not a pass per specialist with entropy-based selection, because the user names the $\Gamma$ they want.

For the policy setting, the nuisance models are estimated once over all quantile levels. A monotone quantile network replaces the per-$\Gamma$ quantile regressions of the sharp estimator. A $\Gamma$-conditioned policy is then trained on the closed-form sharp lower bound with its efficient-influence-function correction, again with $\Gamma$ sampled per batch. No inner adversary is needed here: the closed form already eliminates it, and amortization removes the remaining outer refit.

The selection question is not solved by any of this, because $\Gamma$ is not identifiable from data (Model-Free and Model-Based Policy Evaluation when Causality is Uncertain). What becomes cheap is the decision-relevant summary. The first summary is the breakdown $\Gamma^*$, found by root-finding on the amortized model. The second is a comparison of $\Gamma^*$ against benchmark budgets implied by observed covariates, using the calibration procedures reviewed in Calibrating confounding strength in sensitivity models for weighting estimators. That paper also cautions that such benchmarks frame but do not verify the true confounding strength, and we report them in that spirit. This addresses the dilemma in Confounding-Robust Deferral Policy Learning, where too large a $\Gamma$ makes policies needlessly defer: analysts can see exactly where along $\Gamma$ that over-conservatism begins.

Two motivating precedents show that amortizing over a sensitivity knob is feasible in other settings. Sensitivity-Aware Amortized Bayesian Inference conditions on prior and likelihood context variables and evaluates 162,000 configurations in about 415 minutes after one training. Adversarially Pretrained Transformers May Be Universally Robust In-Context Learners pays the min-max cost once and reuses it across tasks. Neither handles constrained, worst-case causal objectives. We do not build on CausalPFN, which amortizes effect estimation but assumes no hidden confounding. We also do not build on sensGAN (High-Dimensional Sensitivity Analysis for Genomic Studies), whose generator-based adversary is a harder, unstable target that we leave for follow-up once the simpler closed-form cases are settled.

Baselines

The naive reference point trains the unconstrained predictor or plug-in policy once and computes sharp bounds post hoc at every $\Gamma$. It costs one run and answers "how bad could this fixed model be." It does not answer "what is the best model at this budget." Beating it means a lower task loss at a matched worst-case unfairness, or a higher worst-case policy value, at each $\Gamma$.

The strongest competitor, and the quality target, is independent per-$\Gamma$ training. For fairness this is the augmented-Lagrangian neural sensitivity model of Causal Fairness under Unobserved Confounding. For policies it is the sharp efficient estimator of Efficient and Sharp Off-Policy Learning under Unobserved Confounding. We run each at a grid of 11 values of $\Gamma$ in $[1,3]$, with five seeds each, the same architecture, and the same per-run hyperparameter budget. For the policy estimator, the paper reports gains of "up to 4x" over prior methods on synthetic data, and lower regret with 1,000 samples than plug-in approaches reach with 2,500. For the fairness model, no reported MSE or unfairness value was found on Lacuna for the US Survey of Prison Inmates; the paper reports only "nearly zero path-specific unfairness with only a marginal increase" in MSE. Matching this baseline means our per-$\Gamma$ metrics fall inside its seed spread. We also evaluate at held-out $\Gamma$ values between grid points, where the grid baseline must interpolate.

The second strong competitor is warm-start continuation: train at $\Gamma=1$, then initialize each larger budget from the previous solution and fine-tune briefly. It gets the same tuning budget, and we sweep its fine-tuning length to trace a compute–quality curve.

The ablation that isolates our key ingredient keeps $\Gamma$-conditioning but uses a single scalar multiplier shared across budgets. A second ablation drops conditioning and trains only at the largest $\Gamma$, then uses that model everywhere. All methods report total GPU-hours and wall-clock time.

Why not just…?

Why not just compute post-hoc bounds on one model? For pure evaluation that is the right answer, and we use it. But the fairness and policy problems ask for a different model at each budget. A model trained without the constraint can sit far from the constrained optimum, which is why the neural sensitivity framework enforces constraints during training rather than auditing afterwards. The naive baseline comparison measures this gap directly. If the gap is negligible on our benchmarks, the learning case does not need solving and we will report that.

Why not just warm-start a grid? This is the real threat. Continuation exploits the same smoothness in $\Gamma$ that amortization does, and short fine-tunes may cost little. We expect continuation to accumulate drift and to require choosing grid resolution in advance, while the amortized model answers arbitrary $\Gamma$ and makes root-finding for $\Gamma^*$ free. That is an expectation, not evidence. The compute–quality curve settles it. If continuation reaches equal per-$\Gamma$ quality and equal $\Gamma^*$ accuracy at equal or lower compute, we will recommend continuation and report a negative result for amortization.

Why not just interpolate a coarse grid? Interpolating three to five independent runs is cheaper still. However, constrained optima can change non-smoothly when the constraint begins to bind at some intermediate $\Gamma$. The held-out-$\Gamma$ evaluation and the breakdown-point error test whether interpolation misses these kinks.

Proposed Experiments

On synthetic structural causal models with a hidden confounder of known strength, used in all three key papers, we vary the true confounding strength. This lets us check that the amortized frontier covers the oracle unfairness or policy value at the true $\Gamma$, and that the estimated $\Gamma^*$ matches the oracle breakdown. For fairness we use the US Survey of Prison Inmates (2016) task from the neural sensitivity paper. For policy learning we use IHDP and the International Stroke Trial with masked covariates, increasing the number of treatment arms as in the sharp-bounds paper.

The metrics are: per-$\Gamma$ task loss and worst-case unfairness or policy value relative to independent training; constraint violation; error in $\Gamma^*$; and total compute.

The decisive checks are three. First, the amortized model must lie within seed noise of independent training at both grid and held-out $\Gamma$, at no more than about three times the cost of one run. Second, the shared-multiplier ablation must show systematic constraint violation at large $\Gamma$; if it does not, the multiplier network is unnecessary and we will say so. Third, warm-start continuation must not dominate on the compute–quality curve. If extreme budgets underfit because one network cannot cover the whole range, analogous to the clean-versus-robust accuracy trade-off seen in adversarial training, we will report the range over which amortization is reliable rather than average it away.