One Doubly Robust Test for Sparse and Dense Treatment Effects on Many Outcomes

Observational studies now routinely measure hundreds of outcomes per person: glucose readings every few minutes, cytokine panels, microbiome abundances. When asking whether an exposure changed any of them, analysts face two problems at once. First, treated and untreated people differ in their baseline covariates. Handling that requires confounder adjustment: modeling who gets treated and how outcomes depend on covariates, so the comparison is fair. Second, the effect may be dense, meaning small shifts spread over many outcomes, or sparse, meaning a large shift in one or two outcomes. Tests that sum squared evidence over all outcomes catch dense effects and miss sparse ones. Tests that take the single largest deviation do the reverse. Existing confounder-adjusted tests pick one side of this divide. An Efficient Doubly-Robust Test for the Kernel Treatment Effect uses a global kernel distance that behaves like a sum of squares. Identifying sparse treatment effects in high-dimensional outcome spaces is built only for sparse effects in randomized trials.

We propose a single test that does both under observed confounding. For each outcome coordinate we compute a cross-fitted doubly robust score, which stays correct if either the treatment model or the outcome model is right. From this score vector we build a ladder of statistics running from "largest squared deviation" to "sum of all squared deviations". We calibrate the whole ladder jointly with one multiplier bootstrap applied to the per-person scores, so no parametric model of the null data is required. Put simply: adjust for confounding once, at the level of each person's contribution to the effect estimate, then reuse those same contributions both to search across sparsity levels and to price that search.

Core Claim

Let $T_i\in\{0,1\}$ be treatment, $X_i$ covariates and $Y_{ij}$ outcome coordinate $j=1,\dots,p$ for person $i$. With cross-fitted propensity $\hat e$ and outcome regressions $\hat\mu_{1j},\hat\mu_{0j}$, the augmented inverse-propensity-weighted (AIPW) score is

$$\hat\varphi_{ij}=\hat\mu_{1j}(X_i)-\hat\mu_{0j}(X_i)+\frac{T_i\,(Y_{ij}-\hat\mu_{1j}(X_i))}{\hat e(X_i)}-\frac{(1-T_i)\,(Y_{ij}-\hat\mu_{0j}(X_i))}{1-\hat e(X_i)} .$$

The null hypothesis is that the average treatment effect is zero on every coordinate. Let $Z_j$ be the studentized mean of $\hat\varphi_{\cdot j}$, and let $Z_{(1)}^2\ge Z_{(2)}^2\ge\dots$ be the ordered squares. The ladder has two parts. The top-$k$ statistics are

$$S_k=\sum_{l=1}^{k} Z_{(l)}^2, \qquad k\in\{1,2,4,\dots\},$$

so $S_1$ is the max-norm statistic. The dense end, $S_p$, is replaced by a cross-fold product $\sum_j \bar\varphi^{(a)}_j\bar\varphi^{(b)}_j$, where $\bar\varphi^{(a)}_j$ and $\bar\varphi^{(b)}_j$ are means over two disjoint folds. This product has mean zero under the null. It avoids the diagonal noise term that otherwise grows with $p$, which is the accumulation problem described by Two sample tests for high-dimensional covariance matrices. The final statistic is the minimum over $k$ of each rung's p-value. We calibrate it by recomputing every rung on the same Gaussian multiplier draws of the centered scores, which preserves their dependence.

The claim has three parts. The test holds nominal size under confounding, with $p$ growing faster than $n$, when the nuisance models converge at the usual product rate with a $\log p$ penalty. Its power tracks the better of the sparse and dense oracles across the whole sparsity range. It does this without splitting the sample for selection and without simulating from a model of the data-generating process.

How This Relates to Prior Work

Two papers motivate the gap by each covering one side of it. An Efficient Doubly-Robust Test for the Kernel Treatment Effect combines AIPW with a kernel mean embedding and a cross U-statistic that is asymptotically normal, removing the need for permutations. We reuse its cross-fold construction for the dense rung. We differ in adding the sparse rungs, and in targeting coordinate-wise mean effects on a vector outcome rather than general distributional change. Its sensitivity to variance-only shifts is something our test gives up, and we say so. Identifying sparse treatment effects in high-dimensional outcome spaces selects the outcome subset that maximizes Hotelling's $T^2$ with one Lasso call, then tests that subset on a held-out split. Its authors state it is designed for sparse effects in randomized trials. We replace selection-plus-splitting with a jointly calibrated search, and we replace randomization with doubly robust adjustment.

Three papers supply the method. Two-Sample Tests for Inhomogeneous Random Graphs in Lr Norm: Optimality and Asymptotics proves that low-order norms are minimax-optimal for dense differences and high-order norms for sparse ones; that result justifies the ladder rather than a single norm. Computationally efficient and data-adaptive changepoint inference in high dimension shows that data-adaptive combinations over several norms keep power across change patterns. We carry that idea from changepoints to causal effects. Unified Covariate Adjustment for Causal Inference supplies the double/debiased machine learning logic: cross-fitting plus a score that is insensitive to small nuisance errors. We extend that logic from point estimation to a max-type and a sum-type test over $p$ coordinates.

One paper both motivates and constrains the design. A Unified Combination Framework for Dependent Tests with Applications to Microbiome Association Studies fuses a sparse Higher Criticism test with a dense kernel test. It restores size by learning their joint null through a parametric bootstrap from an assumed count model. It shows that ignoring the dependence inflates error: standard Fisher and Stouffer combinations reached 8–9% at a nominal 5%. That result tells us calibration cannot be skipped. Our departure is that the bootstrap acts on estimated influence scores, not on simulated data. It therefore never requires a correctly specified outcome distribution, and it never breaks the link between confounders and exposure the way permuting treatment would.

Research Program

The theoretical work has three pieces.

The first is a size result for the max rung. It requires the sum of nuisance errors, which enters as the product of propensity and outcome-regression errors, to be $o(n^{-1/2}/\sqrt{\log p})$ uniformly over coordinates. The proof combines high-dimensional Gaussian approximation for maxima with the standard AIPW remainder bound.

The second is a size result for the dense rung, and this is where real care is needed. Per-coordinate bias of order $r_n^2$ enters the cross-fold product as a sum of $p$ squared biases. The null standard deviation is of order $\sqrt p/n$. Comparing the two gives a condition roughly of the form $\sqrt p\, r_n^4\, n\to 0$ in addition to the usual $r_n^2\sqrt n\to 0$. We will state this condition precisely and test in simulation where it breaks.

The third is a power result. It should show that the ladder's detection boundary is within a $\log$ factor of the better single rung, as in the adaptive-combination literature.

On the computational side, the propensity model is shared across all $p$ outcomes. For the outcome regressions we compare two options: independent per-coordinate learners, and a multi-output learner that shares covariate features across outcomes, in the spirit of Counterfactual Learning with Multioutput Deep Kernels. We will release an implementation. Once the scores are computed, the bootstrap costs $O(Bnp)$ for $B$ bootstrap draws.

Baselines

The naive reference is a difference in means on each coordinate with no covariate adjustment, combined by Bonferroni. It shows how much confounding alone distorts size.

The strongest existing dense competitor is AIPW-xKTE, run with its published Gaussian kernel and median-heuristic bandwidth. It uses the same cross-fitted nuisance learners and the same number of folds as our method, so any power difference comes from the statistic and not from better nuisance models. The file reports that it "controls the false positive rate around the standard 0.05 threshold" on IHDP and is "roughly 20 times faster" than the earlier kernel test at $n=350$. No reported result was found on Lacuna for its power on high-dimensional vector outcomes, so we measure that ourselves. Beating it means matching its power within Monte Carlo error on dense alternatives while holding size.

The strongest sparse competitor is the Lasso-plus-split method of Jeong, Fox and Johari. We adapt it to observational data by replacing the difference in means on the held-out split with an AIPW estimate on the selected subset, and give it the same learners. No reported numerical power for this method was found on Lacuna; its CGM results are described only qualitatively. Beating it means clearly higher power once more than a handful of coordinates carry effect, and no loss on the sparsest settings.

Two ablations isolate the proposal's key ingredients. The first, and the key one, is a two-rung version: only $S_1$ and the dense rung, combined by Bonferroni on two p-values. This separates what the ladder and joint calibration add from what the doubly robust max and dense components add on their own. The second replaces AIPW scores with plain outcome-regression residuals inside the same ladder, isolating the contribution of double robustness.

Why not just…?

Why not just take the smaller of a doubly robust max-test p-value and a doubly robust sum-test p-value and double it? This is the most serious threat. With only two components, Bonferroni costs at most a factor of two, and max-type and sum-type statistics are often nearly independent in high dimensions, which makes simple combinations close to exact. If the two-rung ablation matches the full ladder, we will report that plainly. The contribution then narrows to the two doubly robust components and their validity conditions. Even the narrowed contribution would be new, since neither AIPW-xKTE nor the Lasso method offers a confounder-adjusted max test. We expect the ladder to help at intermediate sparsity, with 5 to 50 active coordinates out of hundreds. Neither end of the ladder is efficient there, and a Bonferroni correction over many dependent rungs becomes costly. The sparsity sweep in our experiments decides this question.

Why not just residualize outcomes on covariates and feed the residuals to Yu et al.'s combination test? Their calibration simulates null data from a parametric count model. That model says nothing about how treatment was assigned. Residualizing without cross-fitting also lets first-stage fitting error leak into every coordinate. The dense rung then sums that leakage over $p$ coordinates, which is exactly the bias the $\sqrt p\, r_n^4 n$ condition controls. We include this pipeline as a baseline and expect size to inflate as $p$ grows under nonlinear confounding. If it does not, the doubly robust construction is unnecessary in practice, and we will say so.

Why not just run AIPW-xKTE with a narrower bandwidth or a coordinate-wise kernel? A sum of coordinate-wise kernels is still a sum over coordinates, so it inherits the noise accumulation that the minimax result in the $L_r$-norm paper identifies. Shrinking the bandwidth changes which features of the distribution the test weighs, not how evidence is pooled across dimensions. We will nonetheless run xKTE at three bandwidths and with a coordinate-sum kernel on the sparse settings. That tests the argument empirically instead of assuming it.

Proposed Experiments and Decisive Checks

The first benchmark is synthetic. Covariates have 20 dimensions, and the propensity depends nonlinearly on them. There are $p\in\{50,500,2000\}$ outcomes that share covariate-driven confounding, with $n\in\{200,500,1000\}$. The number of affected coordinates ranges from 1 to $p$ at fixed total signal energy, and we also include nuisance misspecification of one model at a time.

The second is semi-synthetic CGM, following Jeong et al. We take real glucose traces, add their 5-unit drop in a 2-hour window for the sparse setting and a small whole-day shift for the dense setting, and assign treatment through a covariate-dependent propensity so that confounding is present. If the 4T traces cannot be obtained, we match their dimension and autocorrelation in simulation.

The third is semi-synthetic IHDP. We keep the real covariates and selective removal of treated infants, and generate 300-dimensional outcomes with a sparse-to-dense sweep of effects.

Finally, we apply the test to the IBD microbiome data. There, Yu et al. report a combined p-value of about 0.020 where the sparse test alone gave 0.210. We ask whether adding covariate adjustment changes that conclusion.

We use four metrics: empirical size at 5% over 2000 null replicates, power curves against sparsity, runtime, and, for CGM, whether the most significant coordinates fall in the true window.

Three checks are decisive. First, size must stay within the Monte Carlo band around 5% at $p=2000$ under misspecification of one nuisance model, and inflate when both are wrong; this confirms the double robustness claim. Second, across the sparsity sweep the power curve must lie within a few points of the pointwise best baseline everywhere, not only at the two ends. Third, the gap between the full ladder and the two-rung Bonferroni ablation at intermediate sparsity measures how much of the proposal survives the simplest alternative.