Abstract
Machine learning models are widely applied in various fields. Stakeholders often use post-hoc feature importance methods to better understand the input features’ contribution to the models’ predictions. The interpretation of the importance values provided by these methods is frequently based on the relative order of the features (their ranking) rather than the importance values themselves. Since the order may be unstable, we present a framework for quantifying the uncertainty in global importance values. We propose a novel method for the post-hoc interpretation of feature importance values that is based on the framework and pairwise comparisons of the feature importance values. This method produces simultaneous confidence intervals for the features’ ranks, which include the “true” (infinite sample) ranks with high probability, and enables the selection of the set of top-k important features.
Introduction
Machine learning models often rank features by importance to provide interpretability. However, these rankings can be surprisingly unstable due to data sampling noise. When using "post-hoc" explanation methods like SHAP or Permutation Feature Importance (PFI), stakeholders usually prioritize the rank: is Feature A more important than Feature B?
Because importance values are calculated from a sample of data, picking a different sample can cause values to shift and rankings to flip—a phenomenon known as ranking instability. As shown in Figure 1, the relative positions of features like 'Workingday' and 'Month' can change between different rankings, potentially leading to inconsistent conclusions.
While methods exist to place "error bars" on importance values, robustly quantifying uncertainty for discrete and relative ranks has remained challenging. Without knowing the uncertainty of these ranks, stakeholders might make critical decisions based on a "Top 5" list that is essentially a product of random noise.
Methodology: The Base-to-Global Framework
To address this, we introduce a framework for Confident Simultaneous Feature Ranking. The core idea is to define global feature importance as the average of "base" importance values. For SHAP, these are local explanations for individual data points; for PFI, they are results of individual permutations.
The method proceeds through three main steps:
- Pairwise Comparisons: Instead of ranking values directly, the method performs pairwise hypothesis tests (e.g., paired-sample t-tests) between every possible pair of features to determine if one is significantly more important than another.
- Simultaneous Confidence Intervals (CIs): To account for the "multiple comparisons" problem, adjustment procedures like Holm’s or Min-P are applied.
- Rank-Set Construction: By counting how many features a specific feature is significantly "better" or "worse" than, the algorithm constructs a range (e.g., "this feature is ranked between 3rd and 5th") that covers the true rank with high probability.
Experimental Setup
The framework was evaluated using both synthetic and real-world datasets:
- Synthetic Data: Two Data Generating Processes (DGP-A and DGP-B) tested varying noise levels and feature correlations.
- Bike Sharing Dataset: Demonstrated ranking stability in a regression context.
- COMPAS Dataset: Analyzed how training set variability affects feature importance in recidivism prediction.
- Nomao Dataset: A high-dimensional dataset (118 features) used to test performance when many features have negligible importance.
Key Results and Insights
- Reliable Coverage: Simulations showed the method maintains simultaneous coverage levels of nearly 100%, outperforming naive bootstrapping approaches which often fail to provide proper coverage.
- Efficiency and Correlation: By using paired t-tests, the method accounts for correlations between feature importance values, leading to tighter confidence intervals than methods assuming independence.
- Top-k Guarantee: The framework allows for identifying a set of features guaranteed (with high probability) to contain the true top-k most important features.
- High-Dimensional Utility: On the Nomao dataset, the method correctly identified 31 features as indistinguishable from one another, signaling their shared low importance.
- Computational Speed: The ranking algorithm is highly efficient, running roughly ten times faster than the computation of SHAP values themselves.
Conclusion
This framework provides a mathematically grounded way to interpret feature rankings with confidence. While currently conservative, future work may refine the intervals for less-than-100% coverage or adapt the method for extremely long-tailed distributions using robust statistics like Wilcoxon tests. By quantifying ranking uncertainty, we move closer to reliable and trustworthy machine learning explanations.
Summary updated