When Hospitals Train at Different Speeds: Doubly Robust Causal Estimates from Unequal Federated Work

Multi-hospital retrospective cohort studies estimate the effect of a treatment by comparing patients who did and did not receive it. To make that comparison fair, they first model who was likely to be treated (the propensity score) and what outcome each patient would likely have had (the outcome model). Privacy rules keep patient records inside each hospital, so these models are increasingly trained by federated learning: each hospital trains locally, and only model parameters go to a coordinating server. Current federated causal methods assume every hospital does the same amount of local training each round. Real networks contain academic centers with GPU clusters next to community clinics on shared servers. The standard workarounds are to wait for the slowest site or drop it. Waiting is slow. Dropping systematically removes small hospitals' patients, which is exactly the population a multi-site study exists to include.

The optimization literature has tools that let each machine do as much work as its budget allows within a fixed deadline. Applied naively to propensity models, these tools create a specific, predictable bias. When sites run unequal numbers of local steps, the trained model drifts toward the sites that ran more steps. For a propensity model fit across hospitals, that drift is not just generic noise. It is equivalent to fitting the propensity score on a re-weighted mixture of hospitals. That is the same error that site "membership weights" were introduced to fix. The proposal is to make heterogeneous local work safe for treatment-effect estimation by treating this bias as a known re-weighting. We correct it with one cheap exact round, and we rely on doubly robust estimation so that the remaining optimization error affects the treatment-effect estimate only at second order.

Core Claim

Let site $k$ hold a fraction $p_k$ of patients and run $\tau_k$ local gradient steps per round. Under small step sizes and standard averaging, the aggregated update behaves to first order like $\sum_k p_k \tau_k \nabla L_k$, where $L_k$ is site $k$'s local loss. The fixed point therefore minimizes the step-weighted objective $\sum_k p_k \tau_k L_k$ rather than the intended $\sum_k p_k L_k$. This is the "objective inconsistency" the file lists as an open problem for variable local workloads.

For a propensity model, the true pooled propensity across sites is

$$e(X) = \sum_k P(S=k \mid X)\, e_k(X),$$

where $S$ is the site label and $e_k(X)$ is site $k$'s treatment probability given covariates $X$. Minimizing the step-weighted objective yields the same formula with the membership probabilities $P(S=k\mid X)$ replaced by tilted ones proportional to $p_k \tau_k P(X \mid S=k)$. Heterogeneous compute therefore acts as a known distortion of the site mixture, because the server knows every $\tau_k$.

The central claim has two parts. First, this distortion causes measurable bias in inverse-propensity estimates of the average treatment effect (ATE) whenever compute budgets correlate with hospital case mix, which is the realistic case. Second, an estimator that (a) lets sites train under a deadline with drift control, (b) estimates membership weights and undoes the known $\tau$-tilt, and (c) finishes with one synchronous, exact pass in which each site computes its sum of doubly robust per-patient scores, matches the pooled-data ATE and its confidence interval coverage at a fraction of the wall-clock time of synchronous training.

The doubly robust estimator used here is augmented inverse propensity weighting (AIPW), which combines the propensity and outcome models so that the estimate stays consistent if either one is right. The reason the final exact pass matters is that AIPW's error depends on the product of propensity error and outcome error. Imperfect local optimization is therefore tolerable as long as both nuisance models are reasonably close, and the score sum itself is computed exactly at each site.

Relation to Prior Work

The closest work is Federated Causal Inference on Multi-Site Observational Data via Propensity Score Aggregation. It introduces Fed-IPW and Fed-AIPW, which aggregate local propensity scores through membership weights $P(S=k\mid X)$ learned as a site classifier, prove pooled-data efficiency, and match a pooled benchmark on the 14-center Traumabase registry. It trains every nuisance model with synchronous FedAvg. Our proposal supplies the method, membership weights, for a problem it does not consider: the distortion that unequal local work introduces. The observation that objective inconsistency is a membership-weight error is what connects the two directions.

Federated Causal Inference in Heterogeneous Observational Data aggregates parametric propensity and outcome likelihoods by Hessian weighting on MarketScan and Optum claims. It constrains the proposal. For parametric logistic models, one-shot Hessian aggregation largely removes the need for iterative training, so our contribution only matters for flexible nuisance models, and we test that boundary directly. Federated causal inference from observational data and Federated Estimation of Causal Effects from Observational Data give Gaussian-process estimators with uncertainty. They note the cubic cost of matrix inversion, which is itself an argument for letting heavy computation run where hardware allows. Both assume uniform rounds.

On the optimization side, On the Convergence of Federated Optimization in Heterogeneous Networks (FedProx) adds a proximal pull toward the global model so clients can do partial work. FedADMM adds per-client dual variables that remember past disagreement and counter drift under inexact local solves. These supply the local training rule. Both were evaluated only on predictive accuracy, never on whether a downstream causal estimate is biased.

Federated Learning in Multi-Center Critical Care Research found on eICU that more local epochs (up to 200) improved mortality AUC. This apparently contradicts the drift concern. We address the contradiction head on. That study used uniform epochs, where step-weighting is a no-op because all $\tau_k$ are equal, and it measured predictive AUC, which is insensitive to a re-mixed site population. Our claim concerns unequal $\tau_k$ and a calibration-sensitive target, so the two findings can both hold. Testing this is one of our decisive checks.

Research Program

We build three components.

The first is a deadline-based federated trainer for the three nuisance models: propensity, outcome regression, and site membership. Each site runs as many FedADMM or FedProx local steps as its simulated budget allows and reports $\tau_k$ alongside its parameters.

The second is a tilt correction. The server receives the membership classifier together with the known $\tau_k$. It forms the correct pooled propensity by recombining site propensities with untilted membership weights rather than trusting the directly averaged propensity model. A simpler variant re-weights each site's update by $1/\tau_k$ during training.

The third is a final exact round. The nuisance models are frozen and broadcast. Each site computes its local sum of AIPW scores and their squares, using cross-fitting across rounds or folds to avoid overfitting bias. The server forms the ATE and a sandwich variance from these sums.

On the analysis side, we will derive the first-order tilt result for strongly convex losses such as logistic propensity, in the style of DISH-type analyses. We will then bound the resulting ATE bias for IPW, which is first order in the tilt, and for AIPW, which is second order. This bound predicts when correction matters: under large dispersion in $\tau_k$, strong correlation between $\tau_k$ and site case mix, and weak overlap.

Baselines

The naive reference point is meta-analysis. Each hospital estimates its own ATE by local AIPW, and estimates are combined by inverse-variance weighting. It is run on identical data with no communication budget. Fed-(A)IPW reports that it fails outright when sites lack local overlap. We include it to confirm that our synthetic designs reproduce that failure. Beating it means lower bias in the no-local-overlap setting, which we expect to be easy.

The strongest existing method is Fed-AIPW with membership weights trained by synchronous FedAvg, where every site waits for the slowest. We give it the same nuisance architectures, the same total number of server rounds, and the same hyperparameter search budget. It reaches the pooled-data answer in its paper, but no reported numerical ATE result was found on Lacuna for Fed-AIPW on Traumabase. We therefore re-run it ourselves. Our method beats it if it matches its bias and 95% interval coverage (within Monte Carlo error) while using substantially less wall-clock time under skewed compute budgets. Matching it at equal wall-clock is not the goal. The variant of this baseline that drops sites missing the deadline gives the "fast but biased" reference.

A parametric competitor is Fed-IPW-MLE with Hessian aggregation from Federated Causal Inference in Heterogeneous Observational Data, run with logistic nuisance models. It has no iterative training, so compute heterogeneity barely matters for it. We expect it to be as good as anything when the true propensity is close to logistic and worse when it is nonlinear. It is the baseline most likely to make our method unnecessary, so it is run on every benchmark.

The optimization reference is FedProx-trained Fed-AIPW without tilt correction. Its paper reports "an average of 22%" absolute accuracy gain over FedAvg with 90% stragglers (FedProx). That is evidence it handles stragglers predictively, not causally. Two ablations isolate the key ingredient. One removes the tilt correction while keeping the exact final round. The other keeps the tilt correction but replaces AIPW with plain IPW. If the first ablation matches the full method, the correction is unnecessary. If the second does, double robustness is not doing the work.

Why not just…?

Why not just wait for the slowest hospital? Synchronous Fed-AIPW is statistically sound, and we expect it to be unbiased. Its cost is wall-clock time, which grows with the slowest site's budget. The question is accuracy per unit of time, so we report bias and coverage as a function of elapsed simulated time under compute skews of 10× to 100×. If the waiting cost is negligible because nuisance training is tiny, the proposal's value shrinks to the flexible-model regime, and we will say so.

Why not just use logistic regression with one-shot Hessian aggregation? This is a real threat. For low-dimensional, near-logistic propensities, Fed-IPW-MLE needs essentially no iterative work, which makes heterogeneity irrelevant. The proposal is only worthwhile when flexible nuisance models are required: nonlinear assignment on high-dimensional EHR covariates, or survival-type outcomes like those that needed neural networks in A Federated Cox Model with Non-Proportional Hazards, where the neural model reached C-index "approximately 66.7" versus "63.5" for linear on METABRIC. Our semi-synthetic designs vary assignment nonlinearity explicitly so the crossover point is measured rather than assumed.

Why not just divide each site's update by its step count? Normalizing updates by $\tau_k$ removes the first-order tilt during training and is simpler than post-hoc membership correction. It is the in-training variant of our correction and is included as an ablation. It may well suffice. If it does, the contribution reduces to showing that this normalization is necessary for causal targets, even though the eICU uniform-epoch results suggest drift is harmless. We consider that worthwhile but smaller. Post-hoc correction can still win when drift control, through proximal or dual terms, interacts nonlinearly with normalization, and the experiments will show which is the case.

Why not just go fully asynchronous? Stale updates destabilize training, and MC-NOMA wireless FL reports deadline-based flexible aggregation converging "over 40%" faster than asynchronous alternatives. Asynchrony also makes the effective site weighting time-varying and hard to correct. We include one asynchronous run as a sanity reference rather than as a main competitor.

Proposed Experiments

The first benchmark replicates the synthetic mixture design of Fed-(A)IPW. It has sites with no local overlap and known ATE. Compute budgets are assigned either independently of case mix or correlated with it, for example older patients concentrated at slow sites. The decisive check is that uncorrected heterogeneous training shows ATE bias that grows with the $\tau$–case-mix correlation, as predicted, while the corrected estimator stays within Monte Carlo error of the pooled oracle and keeps near-nominal 95% coverage.

The second benchmark partitions IHDP into sites, following the multi-source setup of FedCI. That work reports $\sqrt{\text{PEHE}}$ falling from 3.71 with one source to 1.99 with five on DATA-1. We report ATE error and PEHE under heterogeneous budgets.

The third and most realistic benchmark uses real eICU covariates (Apache variables, age) across its 208 hospitals with semi-synthetic treatment assignment and outcomes, so the true effect is known. Compute budgets are tied to hospital size, mimicking the small-clinic regime that Mehrjou et al. identified as fragile. This is where we test the "More is More" contradiction. We will check whether uniform large $E$ is harmless for ATE while unequal $E$ is not.

Across all three benchmarks the metrics are ATE bias, RMSE, interval coverage and width, and simulated wall-clock time. The result that would sink the proposal is uncorrected FedProx or FedADMM training already achieving pooled-level bias and coverage under correlated budgets, or logistic Hessian aggregation matching everything on nonlinear designs. Either outcome is informative and would be reported as the main finding.