Confident Feature Ranking

Bitya Neuhof, Yuval Benjamini

Abstract

Machine learning models are widely applied in various fields. Stakeholders often use post-hoc feature importance methods to better understand the input features' contribution to the models' predictions. The interpretation of the importance values provided by these methods is frequently based on the relative order of the features (their ranking) rather than the importance values themselves. Since the order may be unstable, we present a framework for quantifying the uncertainty in global importance values. We propose a novel method for the post-hoc interpretation of feature importance values that is based on the framework and pairwise comparisons of the feature importance values. This method produces simultaneous confidence intervals for the features' ranks, which include the "true" (infinite sample) ranks with high probability, and enables the selection of the set of the top-k important features.

Introduction: The Problem of Ranking Instability

In machine learning, post-hoc methods like SHAP or Permutation Feature Importance (PFI) are standard tools for identifying which inputs drive a model's predictions. However, these importance values are point estimates derived from finite samples. A significant issue in practice is that these rankings are notoriously unstable; a simple reshuffle or a different subset of data can cause features to flip-flop in their relative importance.

Demonstration of ranking instability where feature orders shift between different samples.

While current uncertainty quantification methods often focus on the importance value itself (e.g., a mean value with a standard deviation), stakeholders typically care about the rank (e.g., "Is Feature A more important than Feature B?"). Because ranks are discrete and relative, standard confidence intervals do not translate well. This paper addresses this gap by providing a statistical framework to calculate simultaneous confidence intervals for feature ranks.

The Base-to-Global Framework

The authors propose a "base-to-global" decomposition that leverages the structure of common importance measures. Most global importance metrics are averages of "base" values—for SHAP, these are local SHAP values per observation; for PFI, these are results from individual permutations.

The proposed pipeline for deriving observed ranks from base feature importance values.

By treating these base values as a sample, the method performs pairwise hypothesis tests (such as paired t-tests) between every possible pair of features. It asks whether the mean importance of one feature is significantly different from another. To maintain statistical rigor across the entire feature set, the method applies multiple testing corrections (like Holm’s procedure or a bootstrap-based Min-P procedure) to control the Family-Wise Error Rate (FWER).

Key Insights and Methodology

This approach produces simultaneous confidence intervals, meaning that with a single probability (e.g., 95%), the true ranks of all features are contained within their respective intervals.

One of the most practical outcomes is the "Top-K" guarantee. The authors prove that if a researcher selects all features whose rank confidence interval overlaps with the top-k positions, they are mathematically guaranteed (with high probability) to have captured the "true" top-k features. This is particularly effective when features are correlated, as the paired tests account for shared variance, leading to tighter intervals than previous ranking methods.

Results and Evaluation

The method was evaluated using synthetic data and real-world datasets, including the Bike Sharing dataset (regression), the COMPAS dataset (classification), and the high-dimensional Nomao dataset.

  • Validity: Simulations confirmed that the method consistently maintained simultaneous coverage at or above the requested levels (90% or 95%), whereas naive bootstrapping often failed.
  • Efficiency: The "Min-P" adjustment was found to be more efficient than the "Holm" adjustment, producing narrower intervals when features were highly correlated.
  • Visualization: Using the Bike Sharing dataset, the authors demonstrated that with small samples (e.g., 50), the rank of a feature like "Workingday" was highly uncertain (CI: [3, 5]), but became precise as the sample size increased to 1,000.

Conclusion and Future Work

While the method is robust, it can be conservative, sometimes providing wider intervals than strictly necessary. Additionally, the reliance on paired t-tests assumes a degree of normality; for datasets with extreme outliers or "heavy tails," robust non-parametric alternatives may be required. Despite these considerations, the framework provides a vital safety net for model interpretability, allowing practitioners to move beyond unstable point estimates toward statistically confident feature rankings.

Summary updated

Abstract

Machine learning models are widely applied in various fields. Stakeholders often use post-hoc feature importance methods to better understand the input features' contribution to the models' predictions. The interpretation of the importance values provided by these methods is frequently based on the relative order of the features (their ranking) rather than the importance values themselves. Since the order may be unstable, we present a framework for quantifying the uncertainty in global importance values. We propose a novel method for the post-hoc interpretation of feature importance values that is based on the framework and pairwise comparisons of the feature importance values. This method produces simultaneous confidence intervals for the features' ranks, which include the ``true'' (infinite sample) ranks with high probability, and enables the selection of the set of the top-k important features.
BibTeX
@inproceedings{neuhof2024confident,
  author = {Bitya Neuhof and Yuval Benjamini},
  title = {{Confident Feature Ranking}},
  booktitle = {International Conference on Artificial Intelligence and Statistics},
  year = {2024},
  url = {https://openreview.net/forum?id=pYGSWGyh1v},
}
Figure