Fairness Thresholds That Survive Unseen Environments

A common way to make a trained classifier fair is to leave the model alone and give each demographic group its own decision cutoff, chosen on a held-out validation set so that, for example, true positive rates match across groups. This works well where it was tuned. On ICU mortality prediction, per-group thresholds cut the true positive rate gap from 0.20–0.23 to under 0.03 (A Controlled Study of Fairness Interventions for Temporal Graph Transformers on ICU Mortality Prediction). The weakness is that the guarantee is only as good as the validation set. Group membership and outcomes are both shaped by unrecorded environmental factors such as neighborhood deprivation, care site, region, or exposure history. When the mix of those environments inside a group changes at deployment, the carefully equalized error rates can drift apart again, and nothing in the fitted thresholds warns anyone.

This note proposes treating that drift the way observational epidemiology treats hidden confounders: as something to be bounded with a sensitivity analysis, not assumed away. We choose per-group thresholds that minimize the worst-case fairness gap over every deployment population reachable by reweighting the validation data by at most a factor $\Gamma$. This reweighting stands for an unobserved environment whose within-group mix shifts. We then set $\Gamma$ in interpretable units by benchmarking it against observed covariates, following the logic of Austen plots (Sense and Sensitivity Analysis: Simple Post-Hoc Analysis of Bias Due to Unobserved Confounding). The deliverable is a post-hoc procedure that still needs no retraining. It outputs thresholds plus a statement of the form "the true positive rate gap stays below 0.05 unless an unmeasured environment shifts group composition more strongly than insurance status does."

Core Claim

Post-hoc fairness methods are near-optimal under their own assumptions. Given calibrated scores, group-specific post-processing reaches the best achievable fairness–accuracy tradeoff (Fair and Optimal Classification via Post-Processing). Multiple group constraints can be enforced by acting directly on operating characteristics (Fair Classification by Direct Intervention on Operating Characteristics). Randomizing near the threshold removes discontinuities (Near-Optimal Algorithm for Debiasing Trained Machine Learning Models). All of these methods take the validation distribution as the truth.

Our claim is that environmental confounding makes the chosen thresholds fragile in a predictable way. Among thresholds that tie on nominal fairness, some are much more stable than others. Stability depends on how many of a group's positive cases sit near the cutoff and on how small the group is. A robust objective that prices in a bounded, unobserved environmental shift picks systematically different thresholds and gives up little accuracy. It also holds fairness better under real held-out-environment shifts than nominal thresholds do. The claim is falsifiable: if robust and nominal thresholds degrade equally on held-out states or care settings, the idea fails.

The Mechanism

Let $s(x)$ be the frozen model's score. For each group $a$, let $t_a$ be that group's threshold. Let $P_a$ be the validation distribution of scores and labels within group $a$. An unobserved environment $U$ can change $P_a$ at deployment. We model the deployed distribution as a reweighting $w$ of $P_a$ with $1/\Gamma \le w \le \Gamma$ after normalization. Here $\Gamma \ge 1$ bounds how much any unit's relative weight can change. This is the same kind of bounded-odds assumption that drives the $\alpha$ axis of Austen plots, moved from treatment assignment to environmental composition. Because $U$ is unobserved, $w$ may depend on anything, including the label.

For true positive rate, the worst case then has a closed form. Among group-$a$ positives, the lowest achievable rate puts weight $\Gamma$ on the misses (score below $t_a$) and $1/\Gamma$ on the hits:

$$ \underline{\mathrm{TPR}}_a(t_a) = \frac{h_a}{h_a + \Gamma^2 m_a}, \qquad \overline{\mathrm{TPR}}_a(t_a) = \frac{\Gamma^2 h_a}{\Gamma^2 h_a + m_a}, $$

where $h_a$ and $m_a$ are the validation counts of hits and misses among group-$a$ positives. The robust procedure chooses thresholds that minimize $\max_{a,b}\big(\overline{\mathrm{TPR}}_a - \underline{\mathrm{TPR}}_b\big)$, subject to an accuracy floor, or trades the two off along a Pareto curve. Equalized odds adds the analogous false positive rate term.

This bound is deliberately crude because it lets $w$ depend on the label freely. The second half of the method tightens it. We restrict the reweighting to depend on the label only through a learned environment proxy, and we replace worst-case over $\Gamma$ with a finite-sample confidence band on $h_a, m_a$. Small groups therefore pay for both sampling noise and shift. That second term matters because sparse minority groups are exactly where post-processing is known to overfit (Fair and Optimal Classification via Post-Processing).

The environmental-confounding literature supplies the method for choosing $\Gamma$. Austen plots make an abstract sensitivity parameter meaningful by placing observed covariates on the same axes. They showed that in the blood-pressure study a confounder "only as strong as age" would overturn the conclusion, while in IHDP even the strongest observed covariates fell far below the danger line. We do the same for fairness. For each observed covariate $Z$ that plausibly marks an environment (US state, insurance type, admission source, care unit), we compute the largest within-group reweighting implied by moving between levels of $Z$. That gives a benchmark $\Gamma_Z$. Practitioners then pick $\Gamma$ relative to named covariates rather than as a bare number.

The Mendelian randomization work motivates the shape of the problem rather than supplying the tool. Addressing Instrument-Outcome Confounding in Mendelian Randomization through Representation Learning shows that population structure acts as a hidden environment. It biases standard estimators by up to 0.3 in semi-synthetic All of Us experiments, and it can be removed only by learning an environment-invariant representation across diverse populations. That paper is in-processing and needs raw multi-population data. We take its picture, a predictor contaminated by a component that varies across environments, and ask what a deployer holding only a frozen score and one validation set can guarantee. Yemin Lan et al. show that regional lifestyle can dominate the biological signal in human biomarkers. That supports the premise that environment shifts are large relative to within-environment variation, not a second-order nuisance.

Relation to Counterfactual Post-Processing

A different line enforces counterfactual rather than statistical fairness after training. λ-PCF interpolates between a model's score and a counterfactually fair score (An end-to-end fairness framework based on counterfactual reasoning). CF-CP symmetrizes conformal scores across interventions on the protected attribute (Counterfactually Fair Conformal Prediction). Principal counterfactual fairness flips a minimal set of decisions using doubly robust estimates (Fairness on Principal Stratum). Each needs either an invertible causal model or no unobserved confounding between attribute and outcome. Those are exactly the assumptions environmental confounding breaks. Our proposal does not attempt counterfactual fairness. We keep the statistical criteria practitioners actually deploy and make them honest about hidden environments. That makes this work complementary to these methods: it assumes less structure and promises less.

Research Program

The work has three parts.

The first is the estimator. We build the robust threshold search for equalized opportunity, equalized odds, and demographic parity. The search is a sort-and-scan per group, so it keeps the CPU-seconds cost profile that makes post-processing attractive; Xian et al. report about 2.5 seconds on 116,000 ACSIncome samples for their linear program. We add a randomized variant in the style of the randomized threshold optimizer, because worst-case objectives often favor mixing between two cutoffs.

The second is the benchmarking step that turns covariates into $\Gamma$ values, and a test of whether it is predictive. The test holds out an environment and checks whether the benchmarked $\Gamma$ bound brackets the fairness gap that actually appears.

The third is an analysis question raised by Locati et al.. They found that fixing race disparities pushed age and sex disparities up, and that calibration alone widened gaps. We test whether part of this "whack-a-mole" pattern arises because the unconstrained attribute is correlated with the hidden environment. If so, robust thresholds should reduce the collateral damage, since they stop fitting cutoffs to environment-specific quirks of the validation set.

Baselines

The naive reference is a single global threshold on the frozen score, with and without Platt scaling. It shows how large the problem is. On MIMIC-IV it leaves a true positive rate gap of 0.20–0.23 per Locati et al.

The strongest direct competitor is nominal per-group threshold equalization after Platt scaling, exactly as in Locati et al.. It reports reducing the true positive rate gap to under 0.03 on MIMIC-IV, on in-distribution test data. This is the number to beat, but only on shifted test environments. In distribution, we expect to match it within noise, not beat it. We will also run the LP post-processing of Xian et al. and the multi-constraint method of Jiang and Dobriban as nominal post-processors. For ACSIncome under held-out-state shift, and for Xian et al.'s method on any cross-environment split, no reported result was found on Lacuna, so we will run these ourselves.

All post-processors see the same frozen base model, the same validation split, and the same threshold grid. The base model is LightGBM, which gave the highest AUROC (0.837) on MIMIC-IV in Locati et al., plus one neural model. Any hyperparameter (the accuracy floor, $\Gamma$) is chosen on validation environments only, and every method gets the same number of settings.

A stronger but information-advantaged baseline sets thresholds per group and per observed environment. It is legitimate when the environment is recorded, and it bounds what our method could hope for when it is not.

Two ablations isolate the key ingredient. Setting $\Gamma=1$ with only the finite-sample band tests whether robustness to sampling noise alone explains any gains. Using a $\Gamma$ chosen by cross-validation instead of by covariate benchmarking tests whether the benchmarking itself carries information.

Beating the strongest baseline means a lower held-out-environment fairness gap, averaged over environments and at the 90th percentile, at matched overall accuracy (within one point), with the benchmarked bound covering the realized gap at close to its nominal rate.

Why not just…?

Why not just set thresholds per group and per environment? When the environment is recorded and well populated, this is the right answer, and we include it as an upper reference. It fails in the cases that motivate the proposal. The relevant environment is often unrecorded, as with neighborhood exposure, or legally or practically unavailable at decision time. Locati et al. already flag protected-attribute use at inference as a legal barrier. And the group-by-environment cells become too small, which is where thresholds overfit. Our experiments hide recorded environments on purpose to measure how much of this oracle's benefit the robust method recovers without it.

Why not just re-tune thresholds at each deployment site with a few labels? This is the real threat. Where labels arrive quickly, as with in-hospital mortality, a few hundred local labels may beat any robust rule. We will measure the break-even point: the number of site-specific labeled cases at which local re-tuning overtakes robust thresholds on held-out states and care settings. If that number is small, around a hundred, the contribution shrinks to settings where outcomes are delayed or censored, and we will say so. The robust thresholds remain useful as the default before those labels exist.

Why not just report bootstrap confidence intervals on the validation gap? A bootstrap captures sampling noise under the validation distribution, not a change of distribution. The $\Gamma=1$ ablation is essentially this method. If it matches the full method under real environmental shift, then the environmental-confounding framing adds nothing and the idea is refuted.

Why not just learn an environment-invariant representation and retrain? The representation-learning MR paper shows this can remove environment-driven bias. But it needs raw data from several diverse environments and a multi-objective training run, and it cannot touch a frozen or vendor model. In-training fairness interventions also produced whack-a-mole effects on MIMIC-IV. We treat it as a different operating point, not a competitor, and cite its semi-synthetic results as evidence that the hidden-environment problem is real.

Proposed Experiments

The main benchmark is ACSIncome with US state as the environment. We fit the base model and thresholds on a subset of states, benchmark $\Gamma$ using within-training-state covariates (such as occupation sector and urbanicity), and evaluate race and sex fairness gaps on held-out states. We report the mean gap, the 90th-percentile gap, and accuracy.

On MIMIC-IV, following Locati et al.'s cohort of 56,866 ICU patients and 48-hour mortality task, we treat insurance type, admission source, and care unit as hidden environments. We remove each from the features, construct test sets whose within-group mix of that variable is shifted, and measure the degradation of race, age, and sex true positive rate gaps under nominal and robust thresholds. Adult and Communities & Crime act as secondary checks with region-like splits where available.

Three results decide the question. First, robust thresholds must show a smaller held-out-environment fairness gap than nominal equalization at matched accuracy. Second, the covariate-benchmarked $\Gamma$ must predict the size of real cross-state degradation; this is scored as coverage of the realized gap by the bound, and the bound must beat a cross-validated $\Gamma$. Third, on MIMIC-IV, robust race thresholds must widen age and sex gaps less than nominal ones do. If the first fails, the proposal fails. If only the second fails, the thresholds may still be useful but the interpretable sensitivity statement is not trustworthy, and we would report that plainly.