When analysts estimate a treatment effect from observational data, they often match: each treated patient is paired with similar untreated patients, so the two groups look alike before outcomes are compared. Matching produces an auditable cohort that clinicians and regulators can inspect, which is why it dominates applied work. It also forces a trade-off the field still handles by rule of thumb. Demanding that the groups match closely removes bias but discards units or makes the matching problem infeasible. Loosening the demand keeps the sample but lets bias back in. Integer-programming matching makes the tolerance explicit, but Large Sample Properties of Matching for Balance leaves choosing it as an open problem, and caliper studies settle on a fixed "0.2 standard deviations" (Optimal Caliper Width for Propensity Score Matching of Three Treatment Groups).
We propose to choose the match by minimizing the posterior expected squared error of the effect estimate. The procedure has three parts. First, fit a Bayesian model of how untreated outcomes depend on covariates, on a held-out fold. Second, correct the matched estimate with that model's mean prediction. Third, select integer control weights that balance the groups only along the directions where the fitted model remains uncertain. With this objective the bias-versus-variance weight stops being a free tuning knob. It is the ratio of two estimable quantities: the outcome noise level and the posterior uncertainty. The matching problem is a soft penalty rather than a hard constraint, so it is always feasible, and its output is still a discrete matched cohort with standardized differences one can print.
Core Claim
Existing optimization-based matching balances covariates against a prior notion of which functions might matter. Integer matching for balance uses a fixed dictionary of splines or polynomials with a tolerance $\delta$ (Large Sample Properties of Matching for Balance). Kernel optimal matching uses a worst case over a kernel function space, with one global trade-off $\lambda$ tuned by marginal likelihood (Generalized Optimal Matching Methods for Causal Inference). Once outcome data has informed a model, part of the bias can simply be subtracted off. The remaining risk lives in the posterior covariance of the outcome function, not the prior one. Our claim has two parts. Balancing in the posterior geometry and adding a regression correction yields lower finite-sample error than balancing in the prior geometry, and the gain is largest under limited overlap. The same objective also gives a principled, automatic replacement for the tolerance $\delta$.
Concretely, consider the effect on the treated (ATT). There are $n_t$ treated units, and each control $j$ receives an integer weight $w_j \in \{0,1,\dots,M\}$ with $\sum_j w_j = n_t$. Let $f_0$ be the untreated outcome function, with posterior mean $m$ and posterior covariance kernel $k_{\text{post}}$ computed from a cross-fitted fold. The estimator is the treated-minus-matched-control difference in $Y - m(X)$. Its conditional mean squared error, averaged over the posterior, is
$$R(w) = d_w^\top K_{\text{post}}\, d_w + \frac{\sigma^2}{n_t^2}\sum_j w_j^2 .$$
Here $d_w$ is the signed weight vector. It places $1/n_t$ on each treated unit and $-w_j/n_t$ on each control. $K_{\text{post}}$ is the posterior kernel matrix over all units, and $\sigma^2$ is the outcome noise variance. The first term is the squared bias the correction cannot remove. The second is the variance from reusing controls. Neither term carries a hand-set weight. When the outcome model is confident in some direction, $K_{\text{post}}$ shrinks there, and the optimizer may leave the groups imbalanced in that direction to retain sample. When the model is uncertain, the optimizer must balance. This puts into practice, for discrete matching, the insight from Optimal Covariate Balancing Conditions in Propensity Score Estimation that the right functions to balance are the outcome regressions. The difference is that we balance the uncertainty about those regressions rather than plugging in a point estimate. Plugging in a point estimate is the failure mode that paper itself flags: outcome-model error propagating into the weights.
Relation to Prior Work
Three lines of work motivate the design.
The bias-variance literature treats bias as a budget to spend rather than a constraint to meet. Fine-Grained Dynamic Framework for Bias-Variance Joint Optimization on Data Missing Not at Random shows that inverse-propensity estimators cannot have bounded variance under strict unbiasedness. It sets a per-instance trade-off, but its global weighting coefficients are still tuned by hand, and it addresses recommendation reweighting rather than causal matching. Estimating Shape Distances on Neural Representations with Limited Samples minimizes variance subject to an explicit bias budget in a quadratic program. Experimental Design for Linear Functionals in Reproducing Kernel Hilbert Spaces shows that error is controlled by bias projected onto the target functional rather than the ambient space. It also identifies the gap that appears when continuous designs are rounded to integers, which is the same gap we face. We take from these papers the idea that the bias term should be measured only in directions that reach the target. Our addition is that "directions that reach the target" should be read off a cross-fitted posterior.
On the matching side, Generalized Optimal Matching Methods for Causal Inference is the closest work. It frames matching as minimizing worst-case conditional error over a kernel space and fits the kernel and $\lambda$ by marginal likelihood. Our objective resembles its quadratic program, with $k_{\text{post}}$ in place of the prior kernel. That substitution brings three changes: a regression correction is added, the trade-off weight is fixed by $\sigma^2$ rather than tuned globally, and the solution is required to be integer. Large Sample Properties of Matching for Balance supplies the integer-weight view and the guarantee that a small, well-chosen bias gives efficient estimators. It also supplies the two open problems we target, infeasibility under poor overlap and selection of $\delta$, along with its call for "doubly robust matching." Tuning-parameter-free optimal propensity score matching approach for causal inference removes tuning by imposing a monotone single-index structure. That approach constrains ours: we must not lose to it on LaLonde. We avoid its monotonicity assumption, but in exchange we rely on a well-calibrated outcome posterior.
Finally, Comparing Experimental and Nonexperimental Methods: What Lessons Have We Learned Four Decades after LaLonde (1986)? shows that strong estimators can still fail placebo tests because of unobserved confounding. This bounds what we claim. The method reduces estimation error under unconfoundedness. It cannot repair confounding the data does not record.
Research Program
The core build is a solver for the posterior-risk objective. With a Gaussian-process or Bayesian spline outcome model fitted to control outcomes, $K_{\text{post}}$ is explicit, and minimizing $R(w)$ over bounded integers is a convex mixed-integer quadratic program. For moderate sizes we solve it exactly. For larger sizes we solve the continuous relaxation and round it with a balance-preserving dependent rounding scheme, and we measure the rounding gap against the exact solution. To keep the design stage honest, the outcome model is fit on one fold and the matching is done on the other, with folds swapped and estimates averaged. This avoids the double use of outcomes that would otherwise make the estimator look better than it is. Hyperparameters of the outcome model, meaning length-scales and $\sigma^2$, are fit by marginal likelihood on the training fold only. After that, no further tuning parameter remains in the matching step.
On the analysis side, we will show three things. First, $R(w)$ is the exact posterior Bayes risk of the corrected matched estimator. Second, under consistency of the posterior, the minimizer attains the same $\sqrt{n}$ rate and efficiency as matching for balance with an oracle tolerance. The argument is that the posterior covariance contracts at the outcome-model rate, which plays the role of a shrinking, data-chosen $\delta$. Third, a frequentist confidence interval combining the matched-pair variance with the residual bias bound has nominal coverage. We will also report what happens when the outcome posterior is miscalibrated. A prior-kernel ablation and an inflated-posterior variant give a direct sensitivity check.
Baselines
The naive reference is the unadjusted treated-minus-control difference in means. Its job is to show how much confounding each benchmark contains. Next comes greedy 1:1 nearest-neighbour matching on the logit propensity score with a 0.2 pooled-standard-deviation caliper, the rule recommended by Optimal Caliper Width for Propensity Score Matching of Three Treatment Groups, run with and without post-matching regression adjustment. Matching for balance (Large Sample Properties of Matching for Balance) is run over the same spline dictionary we use for the outcome model. Because its tolerance has no selection rule, we report it at a grid of $\delta$ values, with the best value marked as oracle, and we record its infeasibility rate. Optimal covariate balancing propensity scores (Optimal Covariate Balancing Conditions in Propensity Score Estimation), via its released package, and augmented inverse probability weighting with the same cross-fitted outcome model represent continuous weighting.
The strongest competitor is kernel optimal matching with empirical-Bayes tuning and automatic relevance determination (KOM++) from Generalized Optimal Matching Methods for Causal Inference. It optimizes nearly the same quadratic form and already learns outcome-relevant length-scales. On its IHDP limited-overlap experiment (Example 3) it reports mean squared error "around 0.028-0.030", against about 0.033 for inverse probability weighting and about 0.034 for 1:1 matching. We will rerun KOM++ on identical replications and folds, with the same Gaussian-process family and the same marginal-likelihood budget. We will also run it in two further forms, with its continuous weights rounded by our scheme and with our regression correction added, so that only the prior-versus-posterior geometry differs. Beating it means a lower mean squared error than KOM++ with regression correction on the IHDP limited-overlap replications at matched integer constraints, with non-overlapping bootstrap intervals over replications. Matching its error while delivering integer cohorts and no tuned $\lambda$ counts only as a partial result.
The ablation that isolates the key ingredient keeps everything in our method fixed but replaces $K_{\text{post}}$ with the prior kernel $K$. A second ablation removes the regression correction while keeping $K_{\text{post}}$. Together they separate the effect of "balance where uncertain" from the effect of "subtract what you know."
Why not just…?
Why not just run KOM++ and round its weights? This is the main threat, because KOM++ fits outcome-driven length-scales and already trades balance against weight concentration. Our objective differs in two places. KOM++ balances against the prior kernel, so it spends sample on directions the data has already pinned down. Its single $\lambda$ is chosen globally rather than set by $\sigma^2$. The ablation that swaps prior for posterior kernel, with the correction and rounding held fixed, is the decisive test. If it shows no gain, the contribution reduces to "KOM plus rounding," and we will say so.
Why not just use AIPW or bias-corrected nearest-neighbour matching? Both subtract an outcome model, as we do. Large Sample Properties of Matching for Balance reports AIPW losing to balance-constrained matching in the limited-overlap Redding–Sturm simulation, because inverse weights explode. Bias-corrected nearest-neighbour matching keeps the fixed caliper and does not choose controls to reduce residual bias. A real risk remains: with a good outcome model and decent overlap, they may tie us. We therefore stratify every result by overlap level and expect our gains to be concentrated in the poor-overlap strata.
Why not just balance the fitted outcome function, prognostic-score style, as oCBPS suggests? Balancing a point estimate of $E[Y\mid X]$ is optimal only when that estimate is right. Optimal Covariate Balancing Conditions in Propensity Score Estimation notes that outcome-model error flows straight into the weights. Its LaLonde estimate of $-306$ against the experimental $886$ shows how far that can drift. Our objective balances the uncertainty around the fitted function instead. A misspecified-outcome simulation arm, where the spline basis deliberately omits an interaction, will show whether this protects us.
Why not just pick $\delta$ in matching for balance by cross-validation? There is no held-out ground truth for a treatment effect, so cross-validating $\delta$ needs a surrogate loss. Covariate-balance surrogates ignore the outcome, and outcome-based surrogates bring back the posterior risk we already use. We include the oracle-$\delta$ version precisely to measure how close our automatic choice comes to the best possible tolerance.
Proposed Experiments
The primary benchmark is IHDP in the limited-overlap configuration of Kallus's Example 3. We report ATT mean squared error, the ratio of interval length to coverage, effective number of controls, and solver feasibility rate across replications. ACIC 2016 tests breadth, with results reported per data-generating process, since Really Doing Great at Estimating CATE? documents heavy-tailed error distributions that averages hide. No reported ACIC 2016 number for KOM++ was found on Lacuna, so we run all baselines ourselves. A Redding–Sturm-style simulation probes non-Gaussian covariates with poor overlap. No reported absolute mean squared error was found on Lacuna for it, so the comparison is internal. On LaLonde NSW against the CPS and PSID comparison groups, the reference points are the experimental $886$, the oCBPS $-306$, the GLM-weighting $-1190$, and PAVA matching's roughly $915$. We additionally run the 1975-earnings placebo test from Comparing Experimental and Nonexperimental Methods and treat placebo failure as a confounding diagnosis, not as a defeat of the estimator.
Three outcomes would settle the proposal. The method succeeds if the posterior-kernel version beats both the prior-kernel ablation and corrected KOM++ on IHDP limited overlap. It must also never be infeasible, and its error must land within a few percent of oracle-$\delta$ matching for balance. It fails if the posterior kernel adds nothing beyond regression correction, or if coverage collapses in the misspecified-outcome arm. A further check concerns the integer constraint. The rounding gap, meaning the excess of $R(w)$ over the continuous optimum, should shrink as $n_t$ grows. If it stays large at clinical sample sizes, the claim that we deliver auditable integer cohorts at no cost must be weakened.
References
- Large Sample Properties of Matching for Balance
- Optimal Caliper Width for Propensity Score Matching of Three Treatment Groups: A Monte Carlo Study
- Generalized Optimal Matching Methods for Causal Inference
- Optimal Covariate Balancing Conditions in Propensity Score Estimation
- Fine-Grained Dynamic Framework for Bias-Variance Joint Optimization on Data Missing Not at Random
- Estimating Shape Distances on Neural Representations with Limited Samples
- Experimental Design for Linear Functionals in Reproducing Kernel Hilbert Spaces
- Tuning-parameter-free optimal propensity score matching approach for causal inference
- Comparing Experimental and Nonexperimental Methods: What Lessons Have We Learned Four Decades after LaLonde (1986)?
- Really Doing Great at Estimating CATE? A Critical Look at ML Benchmarking Practices in Treatment Effect Estimation