Understanding Global Feature Contributions With Additive Importance Measures

Ian Covert, Scott M. Lundberg, Su-In LeeNeurIPS 2020 · OpenReview

While "local" interpretability tools like SHAP and LIME have become standard for explaining individual machine learning predictions, understanding a model’s global behavior remains a significant challenge. Existing global methods—such as permutation tests or feature ablation—often fail to account for complex feature interactions. For instance, if two features are perfectly correlated, removing one at a time (ablation) might incorrectly suggest neither is important. Furthermore, the field has lacked a unifying framework, relying instead on a patchwork of heuristics without a clear mathematical definition of global "importance."

To address this, researchers have introduced SAGE (Shapley Additive Global Importance), a model-agnostic framework that quantifies how much each input feature contributes to a model’s overall predictive power.

A Unifying Framework: Additive Importance Measures

The authors first introduce a theoretical framework that unifies existing methods by viewing feature importance as an additive approximation of a "predictive power" function. They demonstrate that common techniques, like feature ablation or univariate predictors, are essentially evaluating specific, limited subsets of features.

SAGE acts as the comprehensive, "global" version of this concept. By applying game-theoretic Shapley values to a global loss function, SAGE considers all possible feature combinations. It calculates the Shapley value of a feature relative to the model's total predictive power (the reduction in risk or loss across the entire dataset).

Handling Interactions and Missing Features

A core strength of SAGE is its ability to correctly identify importance in the presence of redundancy or synergy. If two features are redundant, SAGE splits the credit; if they are complementary, it recognizes their joint value. To evaluate how a model performs without certain features, SAGE marginalizes them out using their conditional distribution.

The authors also establish a strong theoretical bridge between machine learning interpretability and information theory, proving that for an optimal model using cross-entropy loss, SAGE values are equivalent to a weighted average of Conditional Mutual Information. Furthermore, SAGE is invariant to invertible transformations, meaning the scale or units of a feature (e.g., inches vs. centimeters) will not change its importance score.

Key Results and Applications

The authors evaluated SAGE across a diverse range of tasks, including image classification (MNIST), regression (Bike Sharing Demand), and tabular classification (German Credit Quality, Portuguese Bank Marketing).

Comparison of global importance methods on MNIST

In quantitative tests, the sum of SAGE values proved to be a much more reliable proxy for the actual predictive power of feature subsets compared to permutation tests or univariate models. Visually, on datasets like MNIST, SAGE provides a clearer, more centralized map of feature importance compared to the noisy outputs of feature ablation.

Biological Relevance in High-Dimensional Data

SAGE also demonstrated significant utility in high-dimensional biological data. On a Breast Cancer (BRCA) genomic dataset, SAGE successfully highlighted genes with known clinical significance.

SAGE values for breast cancer gene identification

As shown above, SAGE assigned the highest importance values to known BRCA-associated genes (like BCL11A and BRCA1/2), outperforming baseline methods that often assigned noisy or misleading scores to the same features.

Computational Efficiency

Calculating exact Shapley values is exponentially expensive. To make global explanations feasible for large datasets, the authors proposed a sampling-based approximation. This algorithm is two to four orders of magnitude faster than the "naive" approach of averaging local SHAP explanations across thousands of rows.

Limitations and Future Work

While SAGE represents a significant step forward, it does have limitations. A core part of the algorithm involves marginalizing out "missing" features, which requires knowing the conditional distribution of the data. Because this is often unknown, the authors use marginal sampling as a practical proxy. However, this can sometimes misattribute importance to features the model doesn't actually use if they are highly correlated with ones it does.

Additionally, even with the proposed sampling speedup, SAGE still requires a significant number of model evaluations. Future work may focus on making these estimates even more efficient by leveraging model gradients or specific architectures, such as tree-based models.

Summary updated

Abstract

Understanding the inner workings of complex machine learning models is a long-standing problem and most recent research has focused on local interpretability. To assess the role of individual input features in a global sense, we explore the perspective of defining feature importance through the predictive power associated with each feature. We introduce two notions of predictive power (model-based and universal) and formalize this approach with a framework of additive importance measures, which unifies numerous methods in the literature. We then propose SAGE, a model-agnostic method that quantifies predictive power while accounting for feature interactions. Our experiments show that SAGE can be calculated efficiently and that it assigns more accurate importance values than other methods.
BibTeX
@inproceedings{covert2020understanding,
  author = {Ian Covert and Scott M. Lundberg and Su-In Lee},
  title = {{Understanding Global Feature Contributions With Additive Importance Measures}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2020},
  url = {https://openreview.net/forum?id=wdbDxF4SnmW},
}
Figure