While "local" interpretability tools like SHAP and LIME have become standard for explaining individual machine learning predictions, understanding a model’s global behavior remains a significant challenge. Existing global methods—such as permutation tests or feature ablation—often fail to account for complex feature interactions. For instance, if two features are perfectly correlated, removing one at a time (ablation) might incorrectly suggest neither is important. Furthermore, the field has lacked a unifying framework, relying instead on a patchwork of heuristics without a clear mathematical definition of global "importance."
To address this, researchers have introduced SAGE (Shapley Additive Global Importance), a model-agnostic framework that quantifies how much each input feature contributes to a model’s overall predictive power.
A Unifying Framework: Additive Importance Measures
The authors first introduce a theoretical framework that unifies existing methods by viewing feature importance as an additive approximation of a "predictive power" function. They demonstrate that common techniques, like feature ablation or univariate predictors, are essentially evaluating specific, limited subsets of features.
SAGE acts as the comprehensive, "global" version of this concept. By applying game-theoretic Shapley values to a global loss function, SAGE considers all possible feature combinations. It calculates the Shapley value of a feature relative to the model's total predictive power (the reduction in risk or loss across the entire dataset).
Handling Interactions and Missing Features
A core strength of SAGE is its ability to correctly identify importance in the presence of redundancy or synergy. If two features are redundant, SAGE splits the credit; if they are complementary, it recognizes their joint value. To evaluate how a model performs without certain features, SAGE marginalizes them out using their conditional distribution.
The authors also establish a strong theoretical bridge between machine learning interpretability and information theory, proving that for an optimal model using cross-entropy loss, SAGE values are equivalent to a weighted average of Conditional Mutual Information. Furthermore, SAGE is invariant to invertible transformations, meaning the scale or units of a feature (e.g., inches vs. centimeters) will not change its importance score.
Key Results and Applications
The authors evaluated SAGE across a diverse range of tasks, including image classification (MNIST), regression (Bike Sharing Demand), and tabular classification (German Credit Quality, Portuguese Bank Marketing).
In quantitative tests, the sum of SAGE values proved to be a much more reliable proxy for the actual predictive power of feature subsets compared to permutation tests or univariate models. Visually, on datasets like MNIST, SAGE provides a clearer, more centralized map of feature importance compared to the noisy outputs of feature ablation.
Biological Relevance in High-Dimensional Data
SAGE also demonstrated significant utility in high-dimensional biological data. On a Breast Cancer (BRCA) genomic dataset, SAGE successfully highlighted genes with known clinical significance.
As shown above, SAGE assigned the highest importance values to known BRCA-associated genes (like BCL11A and BRCA1/2), outperforming baseline methods that often assigned noisy or misleading scores to the same features.
Computational Efficiency
Calculating exact Shapley values is exponentially expensive. To make global explanations feasible for large datasets, the authors proposed a sampling-based approximation. This algorithm is two to four orders of magnitude faster than the "naive" approach of averaging local SHAP explanations across thousands of rows.
Limitations and Future Work
While SAGE represents a significant step forward, it does have limitations. A core part of the algorithm involves marginalizing out "missing" features, which requires knowing the conditional distribution of the data. Because this is often unknown, the authors use marginal sampling as a practical proxy. However, this can sometimes misattribute importance to features the model doesn't actually use if they are highly correlated with ones it does.
Additionally, even with the proposed sampling speedup, SAGE still requires a significant number of model evaluations. Future work may focus on making these estimates even more efficient by leveraging model gradients or specific architectures, such as tree-based models.
Summary updated