Measuring and Removing Surgeon Selection Bias When Pooling Enhanced Recovery Studies

Enhanced recovery after surgery (ERAS) is a bundle of perioperative practices, such as shorter fasting, opioid-sparing pain control, early eating and early walking, that aims to cut complications and hospital stay. Much of the pooled evidence for it mixes randomized trials with retrospective cohort studies, which compare patients who happened to receive the pathway against those who did not, using existing hospital records. In those cohorts the surgeon often decides who goes on the fast track, and surgeons put fit patients on it. This is confounding by indication: the reason for giving a treatment is tied to the patient's prognosis, so the treatment looks better than it is. Random-effects meta-analysis, the standard way to average effects across studies that may each have a somewhat different true effect, cannot detect this bias, because it only sees each study's summary numbers.

We propose a meta-analysis that treats each cohort study's selection bias as something to estimate. Three kinds of evidence already sit inside a typical ERAS review. The randomized trials give an unbiased anchor. Each cohort's design shows how much discretion surgeons had: a hospital-wide "before versus after adoption" comparison leaves little room for choosing patients, while a concurrent comparison where surgeons picked who got ERAS leaves a lot. Each cohort's own tables give readable signals of channeling: imbalance in baseline risk, and differences in outcomes ERAS should not change, such as operative time or blood loss. A hierarchical model links these signals to the gap between cohort and trial estimates, then corrects the cohorts. The claim is that this correction recovers the trial-anchored effect with far more precision than trials alone, and that it shrinks headline ERAS benefits such as a 59% complication reduction by an amount we can measure.

Core Claim

In mixed-design ERAS meta-analyses, the bias of a retrospective cohort is not a fixed nuisance. It depends on how patients were allocated and how fitness-sensitive the outcome is, and both can be read from published reports. Write $y_{ik}$ for the observed log effect in study $i$ on outcome $k$. Then

$$ y_{ik} = \mu_k + u_{ik} + D_i\,\lambda_k\,(\alpha_{m(i)} + \gamma\, s_i) + e_{ik}. $$

The other symbols are defined as follows. $\mu_k$ is the pooled true effect and $u_{ik}$ the between-study random effect with variance $\tau_k^2$. $D_i$ equals 1 for a cohort study and 0 for a trial. $m(i)$ is the allocation mechanism: era-based, concurrent discretionary, or matched. $\alpha_m$ is the mechanism-specific bias and $s_i$ is a selection signal built from reported baseline imbalance and negative-control outcome differences, with weight $\gamma$. $\lambda_k$ scales bias by how strongly outcome $k$ depends on baseline fitness. It is fixed near zero for mechanical process outcomes such as time to first feeding, and estimated for complications and length of stay. $e_{ik}$ is within-study sampling error.

The trials identify $\mu_k$. Cohorts whose $s_i$ and $m(i)$ vary identify the bias terms, much as a small trial grounds a large confounded dataset elsewhere. For binary outcomes we pool on scales that keep a causal meaning, so that correcting confounding is not undone by aggregation bias.

Research Program

The method borrows from three papers and differs from each. Removing Hidden Confounding by Experimental Grounding uses a small trial sample at the individual-patient level to learn a linear bias function and subtract it from an observational model. We lift that idea to the study level, where only summaries exist. Our bias function is indexed by allocation design and reported signals instead of patient covariates, which also avoids its need for individual overlap.

Bayesian Meta-Analyses Could Be More models one unrecorded decision variable, the Bishop score, as a latent threshold with expert-elicited priors. On six labor-induction studies it moved a standard relative risk of 1.39 to 1.04 (95% CI 0.93–1.18). ERAS allocation depends on many unrecorded factors, and eliciting a joint prior over frailty, hemodynamics and fluid tolerance is impractical. We replace elicitation with data-driven anchoring from trials and measured signals, and we keep a latent scalar "fitness" version of their model as a baseline.

Causal Meta-Analysis shows that averaging log risk ratios is not the causal effect on the pooled population and proposes aggregation from 2×2 tables. It assumes each table is unconfounded, which is exactly what fails for cohorts. We supply the within-study correction and adopt its aggregation for the final estimand.

The motivating evidence is clinical. Application of ERAS in partial nephrectomy pooled 35 trials and cohorts with protocols of 2 to 12 elements. It found complications RR = 0.41 and postoperative stay WMD = −2.88 days, with no adjustment for selection in the cohorts. The vNOTES same-day discharge cohort states openly that healthy, motivated patients were chosen for the fast track. It matched only on age, BMI and cyst features, yet operative time and blood loss did not differ, which is the kind of negative-control signal we exploit.

Two papers constrain the design. Revisiting diabetes risk of olanzapine versus aripiprazole shows that targeted maximum-likelihood estimation with machine-learned components still inverted clinical consensus when clinicians selected on unrecorded cues. The sepsis opioid cohort kept an adjusted hazard ratio of 6.239 after time-varying adjustment. Together they tell us not to trust within-study adjustment alone, which is why the trial anchor is essential.

The work has three parts. First, we build a structured extraction protocol for ERAS cohort reports. It records allocation mechanism and reported baseline covariates, including ASA class, age and nephrometry or tumor complexity, as standardized mean differences. It also records negative-control outcomes, chosen in advance as outcomes ERAS should not causally move: operative time, intraoperative blood loss, and conversion rate. We apply it to the partial nephrectomy evidence pool, and assemble comparable mixed-design pools for gastrectomy, anchored by trials such as immediate versus early catheter removal, and for gynecologic surgery.

Second, we fit the model in a probabilistic programming language with weakly informative priors. Bayesian estimation of $\tau^2$ suits pools where trials per outcome are few, a problem documented in the labor-induction paper.

Third, we validate on simulated and held-out data before making clinical claims.

Baselines

The naive reference point is conventional random-effects pooling of all studies regardless of design, as in the partial nephrectomy meta-analysis. Its reported results are total complications RR = 0.41 and postoperative stay WMD = −2.88 days (Wu et al.). We rerun it on our re-extracted data to confirm we reproduce these figures before comparing anything.

The second baseline is trials-only random-effects pooling. It is unbiased by construction but imprecise, and it is the real competitor. The proposal beats it only if the corrected all-study estimate is consistent with the trials-only estimate on held-out trials, with a credible interval at least a third narrower. Both run on the same extracted data with the same priors on $\tau^2$.

The third baseline is meta-regression with a single design indicator, trial versus cohort, which estimates one average cohort bias. It is the simplest bias-modeling competitor. Our model beats it only if mechanism and signal terms improve out-of-sample prediction of held-out trial estimates.

The fourth is the latent decision-variable model of Klein et al., adapted with a scalar latent fitness and priors elicited from three surgeons. It gets the same compute and data as our model. It is the strongest published method for omitted decision variables in meta-analysis. On its own labor-induction benchmark it reports RR 1.39 → 1.04 (95% CI 0.93–1.18).

Two ablations isolate the key ingredients. One removes the selection signal $s_i$, leaving mechanism-only bias. The other fixes $\lambda_k = 1$ for all outcomes, so bias does not depend on outcome type. If either ablation matches the full model, that ingredient is dropped.

For experimental grounding and causal aggregation, no reported numeric result was found on Lacuna for ERAS data. Those methods enter as components rather than as standalone competitors.

Why not just…?

Why not just restrict the meta-analysis to randomized trials? This is the most serious threat, and we treat it as a live outcome rather than a straw man. Trials-only estimates are unbiased but lose power on rare outcomes such as deep vein thrombosis or urinary leakage. They also come largely from single tertiary centers, the generalizability concern raised for both the nephrectomy pool and the gastrectomy trial. If held-out validation shows our corrected estimates add no precision over trials-only pooling, we will report that. The contribution then becomes a measured estimate of how inflated the cohort literature is, which is still useful to anyone reading pooled ERAS claims.

Why not just stratify by design or add a design covariate? A single cohort-bias term assumes era-based comparisons and surgeon-selected comparisons are equally biased, and that length of stay is as biased as time to first feeding. Both assumptions conflict with how allocation works in the vNOTES cohort, where selection was explicit and individual. Design stratification also does nothing about aggregation bias on ratio scales (Causal Meta-Analysis). The mechanism-only and $\lambda_k = 1$ ablations test this directly. If the design indicator predicts held-out trials as well as the full model, we adopt it.

Why not just apply E-values or Klein's latent model to each pool? E-values say how strong an unmeasured confounder would have to be to explain away an effect, but not how strong it is. The olanzapine study reported E-values of 1.5 to 3.0 and still could not resolve the inversion (Agniel et al.). Klein's latent model rests on elicited priors whose correctness cannot be checked inside the data, and a multivariate ERAS decision rule makes elicitation shakier. Our model replaces elicitation with a trial anchor that can be falsified. The head-to-head comparison of held-out trial prediction settles which approach is more accurate.

Proposed Experiments

The first experiment is semi-synthetic. We simulate 30 to 40 ERAS-like studies with known true effects. Allocation in discretionary cohorts depends on a multivariate latent fitness that partly drives reported covariates and negative controls, and era cohorts carry a secular-trend bias instead. We vary the number of trials from 3 to 12 and the correlation between latent fitness and reported signals. The metrics are bias, root mean squared error, and 95% interval coverage of the pooled effect, compared across all baselines. We expect the advantage of the selection signal to grow with that correlation and vanish when it is zero. A flat curve would falsify the mechanism.

The second experiment is the decisive real-data check, leave-one-trial-out prediction. In the partial nephrectomy pool and the assembled gastrectomy and gynecologic pools, we fit each method without one trial and predict that trial's effect. We score predictive log-likelihood and calibration of the predicted intervals. The naive pool should predict poorly because cohorts pull it toward exaggerated benefit, and the full model should beat the design-indicator model. A complementary check uses the negative-control outcomes. After correction, the pooled effect of ERAS on operative time and blood loss should be near zero. A remaining nonzero effect means the correction missed bias.

The third experiment is a transfer check on the labor-induction pool, assuming the six studies report enough baseline covariates such as parity and gestational age to build $s_i$. We ask whether the signal-based model, without Bishop-score priors, moves the relative risk from 1.39 toward the latent-model value of 1.04. Agreement would show that measured selection signals can substitute for elicitation. Disagreement would point to where elicited clinical knowledge still matters.

Finally, we report the corrected estimates for complications and postoperative stay in partial nephrectomy beside the published RR = 0.41 and −2.88 days. We also report the estimated mechanism-specific biases $\alpha_m$, the first quantitative answer to how much surgeon channeling inflates the observational ERAS literature.