Borrowing Confounding Strength Across Sites for Small Clinical Cohorts

A hospital with 80 patients who received some intervention faces two problems at once. The sample is too small to estimate anything reliably. The records also leave out things that drive both who got treated and how they did, such as later chemotherapy, drug duration, or frailty the bedside doctor noticed but never charted. The usual fix for the first problem is to borrow data from larger registries. That fix quietly makes the second problem worse: it imports those registries' hidden biases while shrinking the error bars. We propose borrowing something different from the large sources. Instead of borrowing only their outcomes, we borrow evidence about how strong the missing confounders tend to be. Clinical sites record different variables. A variable that one registry records and another omits therefore gives a natural, measurable example of how much leaving that variable out moves a treatment-effect estimate.

The core idea fits in two sentences. When a source site records a variable, say adjuvant therapy duration, that the small target site lacks, we can measure at the source how far the effect estimate shifts once that variable is dropped. Pooling these measured "omission shifts" across sites in a hierarchical model gives the small target site a data-anchored range of plausible confounding bias, in place of an unanchored expert guess or an implicit assumption of zero.

Core Claim

For a small target site $t$, write the effect it can estimate from its own recorded covariates as $\hat\tau_t$. The effect we actually want is $\tau_t = \hat\tau_t - b_t$, where $b_t$ is the bias from confounders the site did not record. Following the partial-identification view of Bayesian Inference and Partial Identification in Multi-Treatment Causal Inference with Unobserved Confounding, we express $b_t$ through an interpretable sensitivity parameter $\lambda_t$: the share of outcome variance, and of treatment variance, explained by the omitted factors beyond what is recorded. The data never identify $\lambda_t$ from site $t$ alone. However, at every source site $s$ that records an omitted-at-target variable $Z$, the pair of estimates with and without $Z$ gives an observed $\lambda_s^{Z}$ for that variable. We place a hierarchical prior over $\lambda$ across sites and variable types and update it with these observed omission shifts. The target then inherits a posterior interval for $\tau_t$ that is wide where sites disagree about confounding strength and narrow where they agree.

The claim is empirical and falsifiable. At target sizes of 50 to 200 patients, intervals built this way should cover the true effect at close to their nominal rate. Two kinds of procedures should fail that test in opposite directions. Intervals from transfer methods that assume no unmeasured confounding should under-cover. Intervals from single-site sensitivity analysis with a hand-set bound should either under-cover or be uselessly wide. This method is not assumption-free. It calibrates confounding against the kinds of variables that some site in the network records. Confounding by factors that no site ever records remains an explicit, separately reported residual parameter.

How This Relates to Prior Work

Three papers motivate the gap by showing that sample-efficient transfer ignores confounding. Boosting Transfer Learning with Survival Data from Heterogeneous Domains reweights source patients toward a small target and treats variables missing in one site as informative splits. It shows that heterogeneous feature spaces can be exploited for prediction, but it has no notion of confounding. We reuse its insight that feature heterogeneity is information, and we turn that heterogeneity into a probe of confounding strength. CTRL Your Shift: Clustered Transfer Residual Learning for Many Small Datasets borrows from sites whose prediction errors correlate. Correlated errors can come from shared unrecorded treatment patterns just as easily as from shared biology, and CTRL cannot tell the two apart. Our model instead clusters sites by their measured omission shifts. Causal Inference from Small High-dimensional Datasets does causal estimation with small labeled samples through adversarial transfer and Bayesian output heads. It requires identical feature spaces and assumes all confounders are measured. Our setting relaxes both of those assumptions.

Two papers supply the method and also constrain it. Zheng et al. give the transparent parameterization we build on. They also prove that a seemingly neutral uniform prior over confounding directions collapses bias toward zero in high dimensions. That result rules out the lazy version of our model and forces the prior on $\lambda$ to be set on the partial-$R^2$ scale and informed by data. Their framework is single-cohort and linear. Our contribution is to make the unidentified part partly learnable across sites. Confounding-Robust Deferral Policy Learning bounds unobserved confounding with a marginal sensitivity parameter $\Gamma$ and explicitly treats that parameter as given. Our approach is one way to estimate such a bound from multi-site data, and we will translate the posterior on $\lambda$ into a $\Gamma$ range for comparison.

The clinical studies in the file show why this matters. The gastric GIST study Influence of lymph node removal on the prognosis of high malignancy potential gastric gastrointestinal stromal tumors: Insights from population-based study reports a matched hazard ratio of 0.55 favoring no lymph node removal, while conceding that imatinib duration and margin status were never recorded. The esophageal NSAID study Effect of perioperative flurbiprofen axetil on long‐term survival of patients with esophageal carcinoma who underwent thoracoscopic esophagectomy: A retrospective study reports an overall-survival hazard ratio of 0.49 without neoadjuvant or adjuvant treatment data. Both are exactly the kind of claim whose robustness our method would quantify, provided some other registry records the missing variable.

Research Program

The first component is the estimator. At each site we fit a flexible outcome and treatment model on the site's recorded covariates. For heterogeneous feature spaces we fit the models per site rather than forcing a shared representation. Each site produces an effect estimate with its sampling uncertainty. For each source site and each variable it records that the target lacks, we refit with that variable removed and compute the implied partial-$R^2$ pair. That pair is the strength with which the removed variable explains residual outcome and treatment variation. These pairs become noisy observations of $\lambda$.

The second component is a hierarchical model over $\lambda$. Variables are grouped by clinical role, for example "downstream therapy," "baseline severity," and "social proxy," with the last covering cases like marital status in the SEER pancreatic analysis Marital status independently predicts pancreatic cancer survival in patients treated with surgical resection: an analysis of the SEER database. Sites are partially pooled within groups. The target's $\lambda_t$ is drawn from this posterior predictive, and the target's own small-sample estimate supplies the identified part. We will also derive how much the interval shrinks with the number of informative source sites versus the target size. Target sampling error falls roughly as $1/\sqrt{n_t}$. The sensitivity component falls only with the number of sites, because each site contributes one noisy $\lambda$ draw. So we expect a regime where adding source sites helps more than adding target patients, and we will test that prediction.

The third component is a diagnostic that works on real data without ground truth: leave-one-omission-out prediction. We hold out a source site that records $Z$, predict its omission shift from the other sites' posterior, and check calibration. If this fails, the transfer of confounding strength is not justified for that variable group, and the method says so instead of returning a narrow interval.

Baselines

The naive reference is the target-only estimate with its ordinary confidence interval, which assumes no unmeasured confounding. On confounded data it should under-cover badly, and it sets the floor. For the prediction-transfer view, the cited evidence is that a target-only Cox model reached a C-index of 0.500 on the MAGGIC "Euro" study while Transfer Survival Boosting reached 0.677, as reported in Boosting Transfer Learning with Survival Data from Heterogeneous Domains. That is a discrimination metric, not effect coverage, so it tells us TSB is the strongest borrower of predictive signal, not the strongest effect estimator. We will run TSB-style instance reweighting on the same sites, extract an effect as the treated-versus-untreated contrast in its predictions, and bootstrap an interval. The fair comparison uses the same site partitions, the same target sizes, and the same outcome models. Beating it means nominal coverage where it under-covers.

The strongest causal-transfer competitor is Causal-Batle from Causal Inference from Small High-dimensional Datasets, run on the shared-feature subset so its identical-feature requirement holds. No reported result was found on Lacuna for its exact effect-estimation error on IHDP; the file states only that it beat Dragonnet, augmented inverse propensity weighting, and CEVAE in low-sample regimes. We will therefore rerun it ourselves with an equal tuning budget. The strongest sensitivity competitor is single-site partial-$R^2$ analysis in the style of Zheng et al., with the bound set by the conventional practice of benchmarking against the strongest recorded covariate at the target. We also compare to a marginal-sensitivity interval with $\Gamma$ fixed at values typical of that literature. No reported regret numbers on IST were found on Lacuna for ConfHAI; its results appear only as curves. Two ablations isolate the key ingredient. The first uses independent per-site $\lambda$ with no borrowing. The second uses a single $\lambda$ fully pooled across all sites and variable groups. If full pooling matches the hierarchical model, then the clinical-role grouping is unnecessary. We will judge success by coverage within a few points of nominal together with interval width no larger than the best competitor that also achieves nominal coverage.

Why not just…?

Why not just impute the missing confounder at the target from sites that record it? This is the real threat, and we treat it as a headline baseline. Multiple imputation across sites is simpler, and it would work well if the relationship between recorded covariates and the omitted variable transferred across sites. The failure case is when the omitted variable is assigned by local practice, for example one region's imatinib duration policy. Then imputation reproduces the source policy at the target and returns a confidently wrong estimate. Our semi-synthetic design varies whether the covariate-to-confounder relationship is shared or site-specific. If imputation matches our coverage when the relationship is shared, we will report that honestly. In that case our method's value is confined to the site-specific regime, and the leave-one-omission-out diagnostic tells a practitioner which regime they are in.

Why not just run a standard sensitivity analysis at the target with an expert-chosen bound? Zheng et al. show how sensitive conclusions are to how the confounding prior is specified, and ConfHAI's authors note that a misspecified $\Gamma$ makes policies overconservative or suboptimal. With 50 patients, the usual practice of benchmarking against the strongest observed covariate is itself very noisy. The experiments measure this directly by comparing coverage and width against the hand-set baseline at each target size.

Why not just pool all sites and adjust? CTRL's evaluation shows that global pooling mispredicts the smallest sites. The GIST study shows that matching on recorded covariates still leaves a large, unexplained effect when a key treatment is unrecorded. Pooling shrinks the variance while keeping every site's hidden bias. Our experiments include a pooled adjusted estimator and should show the sharpest under-coverage precisely when bias is shared across sites.

Proposed Experiments

The primary testbed has known ground truth. We take the International Stroke Trial used by ConfHAI, whose randomized treatment arms give a reference effect. We carve it into pseudo-sites and induce observational treatment selection on recorded clinical variables. Then we delete different subsets of those selection variables at different sites, so each site omits a confounder that some other site records. We vary target size over 50, 100, and 200 patients, the number of source sites, and whether confounding strength is homogeneous or site-specific. We apply the same construction to IHDP, and to the larger semi-synthetic GWAS setting where feature heterogeneity is simulated. The metrics are interval coverage of the reference effect, interval width, sign-error rate, and absolute bias of the point estimate. The decisive check is whether our intervals hold nominal coverage at 50 patients while the target-only, TSB-derived, and pooled intervals fall well below it. A second check is whether the hierarchical model is narrower than independent per-site sensitivity analysis at equal coverage. If neither holds, the idea fails.

The real-data application is the MAGGIC heart-failure meta-database. Its 30 constituent studies record different variables, which is exactly the heterogeneity the method needs. We will identify variables recorded in some studies and absent in others, compute omission shifts, and run the leave-one-omission-out calibration check. For small target studies such as Euro, we will report effect intervals for the recorded treatments alongside the TSB predictive baseline. The expected outcome, and the one that would make the method useful to clinicians, is an interval that is honest about how much of a small site's apparent treatment benefit could be explained by the kinds of omissions other sites can actually measure.