TL;DR
FUSHAP solves a major headache in machine learning interpretability: estimating population-level Shapley values across multiple data sources with systematic, blockwise-missing features. Instead of relying on flawed data imputation that warps feature importance, FUSHAP uses an influence function framework to treat partially observed sites as variance-reducing control variates, slashing estimation error by up to 8× while safely screening out incompatible data sources.
What Problem Does This Paper Address?
Shapley value (SV) frameworks, like SHAP and SAGE, are the gold standard for post-hoc model interpretability because they fairly distribute credit to features based on cooperative game theory. However, existing population-level SV estimators assume that your reference data is fully observed under a uniform feature space.
In the real world—such as multi-center clinical trials or distributed sensor networks—this assumption falls apart. Different institutions follow different protocols, leading to blockwise missingness, where entire blocks of features are completely absent from certain sites.
The standard engineering fix is to impute the missing features before running SV estimation. The authors show that this is a dangerous trap: imputation fundamentally alters the underlying covariance structure of the data, introducing systematic, coalition-dependent bias into feature rankings that doesn't disappear even if your predictive model gets more accurate.
Methods
The authors introduce FUSHAP (Fusion Shapley Attribution from Partially-observed data), an imputation-free framework that safely fuses data from three source types: Labeled Complete ($LC$), Labeled Missing ($LM$), and Unlabeled Complete ($UC$, which defines the target population).
[LC Site: Small, Fully Observed] ---> Base Shapley Estimate
|
[LM Sites: Large, Block-Missing] ---> Control Variates (via Influence Function) ---> [FUSHAP Combined Estimate]
|
[UC Site: Large, Unlabeled] ---> Target Population Reference
FUSHAP operates via four main steps:
- Preliminary Estimation: It computes a base, importance-weighted Shapley estimate using only the small, fully observed $LC$ site.
- Influence Function Projection: Rather than filling in missing values, the authors derive the exact influence function of the constrained weighted least squares (WLS) Shapley estimator. They use this mathematical shortcut to project how the partially observed $LM$ sites can serve as control variates to reduce the variance of the base estimate.
- Permutation-Based Screening: To protect against distribution shifts, FUSHAP runs a non-parametric permutation test. If an auxiliary site's data distribution is fundamentally incompatible or misaligned with the target population, it is automatically excluded.
- Total-Variance Calibration: FUSHAP dynamically calculates optimal scalar calibration weights to weight each approved site's contribution based on its informativeness, minimizing the total variance of the final attribution vector.
Datasets
The framework was evaluated using three distinct setups:
- Synthetic Data: Three outcome models of varying complexity (Linear, Polynomial Interactions, and Non-polynomial) were used to rigorously analyze imputation bias and variance reduction.
- Beijing Multi-Site Air Quality Dataset: Real-world environmental data from 12 monitoring stations. The task was to attribute a model predicting PM2.5 concentrations using 10 meteorological and pollutant features, with different stations artificially stripped of specific feature pairs to simulate blockwise missingness.
- NACC Alzheimer’s Disease Dataset: Multi-center clinical data from the National Alzheimer’s Coordinating Center. It uses 9 demographic and clinical features to predict Mini-Mental State Examination (MMSE) scores, featuring fully complementary synthetic missingness blocks across sites.
Key Insights and Ideas
- Imputation is a Trap for Interpretability: The paper proves empirically that switching between standard imputation methods (like mean imputation vs. MICE) does not eliminate attribution bias. Because Shapley values depend heavily on expectations over feature subsets, the artificial dependencies introduced by imputation propagate directly into the final feature rankings.
- Combinatorial Control Variates: FUSHAP elegantly extends variance-reduction techniques from simple scalar estimands to a combinatorial setting ($2^p$ coalitions) by leveraging the linear structure of the WLS Shapley formulation.
- Data Fusion Without Data Completion: FUSHAP demonstrates that auxiliary data does not need to be "completed" to be useful. By understanding how the observed components interact with the estimator's influence function, partially observed data can be harvested purely for variance reduction.
Key Results
- Massive Error Reduction: In synthetic experiments, FUSHAP achieved a 3–8× lower Mean Squared Error (MSE) than the single-site estimator and a 2–3× lower MSE than imputation baselines.
- Real-World Superiority: On the Air Quality and Alzheimer's datasets, FUSHAP reduced MSE by 3–7× relative to the single-site baseline.
- Imputation Backfire: In the Alzheimer’s clinical application, standard imputation actually increased the MSE above the single-site baseline, proving that imputation bias can easily outweigh its variance-reduction benefits.
- High-Power Screening: The permutation-based screening step identified misaligned sites with 82% power under moderate misalignment and 100% power under strong misalignment.
- Computational Efficiency: FUSHAP ran 2.7–4.6× faster than the "impute-then-pool" baseline because it computes Shapley values on the small $LC$ sample size rather than a massive, fully imputed pooled dataset.
Limitations and Future Work
- Complete Target Population Covariates: FUSHAP currently assumes that the unlabeled target population sample ($UC$) has complete feature coverage. Extending the framework to scenarios where the target population itself is only partially observed remains an open challenge.
- Centralized Data Access: The current implementation assumes centralized access to row-level data. However, because FUSHAP’s augmentation and calibration steps rely on site-level averages and covariance summaries, the authors note it is highly amenable to future privacy-preserving, distributed/federated implementations.
Summary updated