Bounding Feature Attributions Under Unmeasured Confounding

When a clinician or scientist fits a predictive model and reads off which features matter, usually through tools like SHAP, a method that splits a prediction into per-feature credit, two kinds of uncertainty sit between the explanation and the conclusion. The first is well studied. Different attribution methods applied to the same model and the same prediction often disagree, sometimes with near-zero rank correlation. The second is mostly ignored by explainability research. A feature can look important only because it stands in for a variable that was never recorded and that drives both that feature and the outcome. This is unmeasured confounding. Recent work treats the first problem as the thing to fix, by forcing explainers to agree during training or by choosing models whose explanations look plausible. Nobody has measured how large the first uncertainty is compared with the second.

This note proposes to put both on the same scale. Under a stated adjustment assumption, the explanation that averages a fitted model over the data, the partial dependence, which is the backbone of interventional Shapley values, is exactly the estimate a causal sensitivity model returns when unmeasured confounding is set to zero. Sharp sensitivity bounds can therefore be wrapped around a standard attribution. Each feature then gets an interval that widens as the allowed strength of hidden confounding grows, plus a "breakdown" strength at which its sign or rank claim stops holding. The empirical bet is this: on realistic observational data, these confounding intervals are much wider than the spread across explainers at modest confounding strengths. If so, consensus methods are polishing the smaller source of error.

Core Claim

Write $X_j$ for one feature, $X_{-j}$ for the remaining features, $\hat f$ for the fitted model, and $q_{25}, q_{75}$ for the 25th and 75th percentiles of $X_j$. Define the model-side importance as the interquartile contrast of the partial dependence:

$$\hat I_j = \frac{1}{n}\sum_{i=1}^n \left[\hat f(q_{75}, X_{-j,i}) - \hat f(q_{25}, X_{-j,i})\right].$$

Suppose $X_{-j}$ blocks all back-door paths from $X_j$ to the outcome $Y$, and no feature in $X_{-j}$ is caused by $X_j$. Then $\hat I_j$ estimates the world-level causal contrast $I_j = E[Y \mid do(X_j=q_{75})] - E[Y \mid do(X_j=q_{25})]$. If an unrecorded variable $U$ also shifts $X_j$ and $Y$, then $I_j$ is not identified.

The Generalized Marginal Sensitivity Model (GMSM) of Sharp Bounds for Generalized Causal Sensitivity Analysis addresses exactly this setting. It bounds how far the density of $X_j$ given $(X_{-j}, U)$ may stray from its density given $X_{-j}$ alone, up to a factor $\Gamma \ge 1$, and it handles continuous treatments. From this it derives sharp bounds on the interventional means. Taking the upper bound at one quantile minus the lower bound at the other gives an interval $[I_j^-(\Gamma), I_j^+(\Gamma)]$. That interval collapses to the partial-dependence attribution at $\Gamma=1$.

The new objects are three:

  1. The per-feature attribution interval.
  2. The breakdown value $\Gamma_j^\ast$, the smallest $\Gamma$ at which the interval crosses zero.
  3. A pairwise rank-robustness value $\Gamma_{jk}^\ast$, the smallest $\Gamma$ at which "$j$ matters more than $k$" can no longer be asserted.

These convert the question "which explainer should I trust?" into "is the disagreement between explainers larger or smaller than what plausible hidden confounding already allows?"

How This Relates to Existing Work

Three papers motivate the gap. Feature relevance quantification in explainable AI: A causal problem argues that interventional attribution is the correct way to explain a model's logic. It shows that conditional attribution commits "guilt by association" when a latent common cause links features, but it stays with linear-Gaussian toy settings. Problems with Shapley-value-based explanations as feature importance measures shows that no Shapley variant both stays on the data manifold and reflects the model faithfully under correlation, and calls for causal integration without supplying an algorithm. A Guide to Feature Importance Methods for Scientific Inference shows that different importance metrics estimate different statistical quantities. We take that further: none of those quantities is the causal one once confounding is unmeasured.

Two lines of work try to resolve disagreement and constrain our design. PEAR, from Reckoning with the Disagreement Problem: Explanation Consensus as a Training Objective, adds a correlation-based agreement penalty to training. EXAGREE, from EXAGREE: Mitigating Explanation Disagreement with Stakeholder-Aligned Models, searches among near-optimal models for one whose explanations match stakeholder priors. Both treat disagreement as the target. Neither asks whether the agreed-upon ranking is causally meaningful, and EXAGREE can encode human beliefs that are themselves confounded.

The closest causal attribution method is Causal SHAP: Feature Attribution with Dependency Awareness through Causal Discovery. It learns a causal graph with the PC algorithm and restricts Shapley sampling to causal descendants. That work assumes every common cause is observed, and its authors name latent-variable discovery as future work. Our method does not learn a graph and does not point-identify anything. It reports how conclusions degrade as hidden confounding grows.

GRANITE: A Generalized Regional Framework for Identifying Agreement in Feature-Based Explanations explains why explainers disagree: they treat interactions and feature dependence differently. It shows that partitioning the input space removes most of this disagreement. We reuse its diagnosis and the four-group explainer taxonomy of Toward Understanding the Disagreement Problem in Neural Network Feature Attribution to place each explainer relative to the $\Gamma=1$ endpoint.

Finally, GMSM supplies the bounding machinery outright; we reuse it rather than invent new bounds.

Research Program

The work has three parts.

Formal alignment. We first state and prove when the partial-dependence attribution of a fitted model coincides with GMSM's $\Gamma=1$ estimand. The key condition is that the features other than $X_j$ include no descendants of $X_j$. We also characterize what breaks when that fails: a mediator among $X_{-j}$ turns the target into a controlled direct effect. GMSM's mediated-treatment extension is the candidate fix here, though that extension itself assumes no hidden mediator–outcome confounding.

We then derive simultaneous intervals across all features, so that a full ranking can be certified, and define $\Gamma_j^\ast$ and $\Gamma_{jk}^\ast$. In this setting $\Gamma=1$ means "no unmeasured confounding of $X_j$." The note does not try to model latent dependence among features that does not touch $Y$. Under the interventional view that dependence does not change the model-side estimand; it only affects the off-manifold evaluation issue raised by Kumar et al.

Building the tool. The tool is a wrapper. It takes any fitted model and dataset, fits a conditional density model for $X_j$ given $X_{-j}$ (normalizing flows, as in GMSM), and returns intervals and breakdown values for every feature. A practitioner needs a way to choose a plausible $\Gamma$. We calibrate it by benchmarking against observed covariates: drop each recorded covariate in turn, measure how much it shifts the density of $X_j$, and treat the largest such shift as a reference scale for "a confounder as strong as the strongest one we measured." This is a heuristic anchor, not a guarantee. We will report it as such and show results across a range of $\Gamma$.

The comparison the field is missing. For each feature, we lay out three spreads on one axis:

  1. Explainer spread: the range of normalized attributions across the four explainer groups.
  2. Rashomon spread: the range across near-optimal models, as in EXAGREE's 5% tolerance.
  3. The confounding interval width at calibrated $\Gamma$.

We then ask which one controls whether a ranking claim survives.

Baselines

The naive reference point is the way practitioners report results today. They fit a multilayer perceptron or gradient-boosted model, compute one explainer (Kernel SHAP or interventional SHAP), and read the ranking as a finding. A second naive reference is the same pipeline run with every explainer in the four-group taxonomy of Toward Understanding the Disagreement Problem in Neural Network Feature Attribution, with each explainer's own ranking reported. Our method beats these if its intervals contain the oracle attribution while single-explainer point rankings miss it. The oracle is the attribution computed with the hidden variable restored.

The strongest existing method for causally faithful attribution is Causal SHAP (Causal SHAP). On its synthetic lung cancer benchmark it reports an RMSE of 0.0167 against ground-truth causal effects, against 3.9193 for Kernel SHAP and 8.8037 for Shapley Flow. It is the strongest competitor because it is the only cited method that targets world-level causal relevance rather than model logic. We will run it with the authors' PC-plus-IDA pipeline and with an FCI substitute. Both versions use the same fitted model, the same data splits, and the same conditional-sampling budget as our density models. Our method beats it if, once a confounder is hidden, the oracle effect falls outside the Causal SHAP point estimate's error band but inside our interval at the calibrated $\Gamma$. On data with no hidden confounder, we must not lose to it badly at $\Gamma=1$.

For disagreement resolution, the baseline is PEAR (Reckoning with the Disagreement Problem). On California Housing it raised pairwise rank agreement between Grad and IntGrad from 87% to 96% at an accuracy cost under two percentage points. We will train it with the authors' consensus weight swept on the same validation budget as our base models. The question is whether rising agreement changes $\Gamma_j^\ast$ or oracle coverage. If PEAR-trained rankings match the oracle as well as our intervals do, our central argument fails.

The point-identification alternative is a substitute-confounder approach. We will use the PriMeD-style conditional variational autoencoder from Mitigating health disparities in EHR via deconfounder. It reports an AUROC of 0.7013 on MIMIC-III against 0.6984 for a vanilla network. We infer a latent variable, add it to the features, and recompute partial-dependence attributions. It gets the same architecture search budget as our models.

The ablation that isolates our key ingredient runs GMSM bounds on effect estimates refit from scratch with a separate outcome model, instead of reusing the predictive model's partial dependence. This tests whether tying the bounds to the deployed model's explanation matters, or whether we have only re-labelled per-feature sensitivity analysis.

Why not just…?

Why not just enforce explanation consensus with PEAR? PEAR reduces disagreement among explainers about what the trained network does. When a feature proxies an unrecorded cause, that agreement simply becomes unanimous on the proxy, which is the guilt-by-association mechanism of Janzing et al.. The proposal tests this directly. We train PEAR and unregularized networks on semi-synthetic data with a planted hidden confounder and compare oracle-rank accuracy and $\Gamma_j^\ast$ between them. We expect agreement to rise while oracle-rank accuracy stays flat. If accuracy also rises, PEAR is a serious competitor and we will say so.

Why not just fit a linear model with covariate adjustment and run classical sensitivity analysis? Residualization makes coefficients equal to adjusted effects only when the true relationships are linear and additive. The synthetic process in Toward Understanding the Disagreement Problem contains piecewise and discontinuous effects, and PEAR reports neural models sitting on a better accuracy–agreement frontier than logistic regression. Still, this is a real threat on smooth data. We include an adjusted linear model with the same sensitivity bounds as a baseline. If it covers the oracle as well as our method on the real-covariate benchmarks, the honest conclusion is that the neural model added nothing there.

Why not just run causal sensitivity analysis feature by feature, ignoring the predictive model? This is the strongest objection, because GMSM is our engine. Our answer is the ablation above. Reusing the deployed model's partial dependence lets us bound the explanation actually shown to users, and lets us place every explainer group on the same axis. A refit effect model cannot do that. If the ablation shows refit bounds are tighter or equally valid, we will recommend that practitioners use them and drop the claim that attribution-tied bounds add value.

Why not just point-identify the effect with a deconfounder or conditional instruments? Substitute-confounder and instrument methods can give point estimates. CIV.VAE in Causal Inference with Conditional Instruments Using Deep Generative Models reports an absolute causal error of 0.22 against 11.06 for two-stage least squares on its synthetic benchmark. But these methods need multiple causes or valid instruments, which feature-attribution settings rarely provide. We will run the deconfounder in settings where its assumptions hold and where they are violated. We expect it to win clearly when they hold, and to give confidently wrong attributions when they fail.

Proposed Experiments

The core testbed hides real or planted confounders so that the oracle is known.

Fully synthetic. We extend the additive synthetic process of Wright and Koenen and the lung cancer graph of Causal SHAP with a latent variable $U$ of controlled strength. $U$ affects one feature and the outcome, and the true $\Gamma$ is computed analytically.

Semi-synthetic. We use the IHDP covariates already evaluated by GMSM.

Real covariates with a held-out column. On California Housing and Bikesharing, we remove a strong covariate (median income for housing, weather for bikes). We define the oracle attribution as the partial dependence of a model trained with that column restored, under the assumption that the restored column completes the adjustment set. On MIMIC-III, we withhold insurance type as a stand-in for the unrecorded socioeconomic status that PriMeD targets.

Metrics. We report:

  1. Oracle coverage of the intervals at the true or calibrated $\Gamma$.
  2. Interval width.
  3. Accuracy of certified pairwise rankings: the fraction of claims certified at $\Gamma$ that agree with the oracle.
  4. The ratio of explainer spread to confounding interval width per feature.

We also report runtime, for comparison with GMSM's 0.4 seconds against 137 seconds for prior continuous methods on its own task. On the OpenXAI datasets (OpenXAI), where no oracle exists, we report only the spread ratio and $\Gamma_j^\ast$ distributions.

Decisive checks. Three results would settle the proposal.

  1. On fully synthetic data, intervals must cover the oracle whenever $\Gamma$ is at least the true value, and must not cover it for smaller $\Gamma$ when the confounder is strong. A failure here means the alignment theorem or its assumptions are wrong in practice.
  2. Across the held-out-column benchmarks, we predict that confounding interval width at calibrated $\Gamma$ exceeds explainer spread for most top-ranked features. If explainer spread dominates instead, consensus methods are aimed at the right target and our motivating claim is false.
  3. PEAR training should raise explainer agreement without improving oracle-rank accuracy or $\Gamma_j^\ast$. If it improves both, we treat consensus training as complementary rather than misdirected.

For context on scale of disagreement-reduction claims: GRANITE reports a 96% reduction in disagreement on Bikesharing through partitioning. We know of no comparable reported figure on Lacuna for oracle coverage of any attribution method under hidden confounding, so these coverage results would be the first of their kind.