TL;DR
This paper introduces a new, computationally efficient method for constructing confidence intervals for Shapley values, which attribute the explained variance in a regression model to individual predictors. This method offers a significant speedup over existing bootstrap techniques and provides reliable estimates, making it a valuable tool for understanding predictor importance.
What Problem Does This Paper Address?
In linear regression, understanding how much each predictor contributes to the overall explained variance (R²) is crucial for inference and model interpretation. While the Shapley value decomposition offers a theoretically sound way to attribute R² to individual predictors, calculating confidence intervals for these values has historically relied on computationally intensive bootstrap methods. This paper addresses the need for a more efficient and practical approach to quantify the uncertainty associated with Shapley value attributions.
Methods
The core of this paper lies in deriving the asymptotic normality of Shapley values for the coefficient of determination. This is achieved by leveraging existing results on the asymptotic distribution of correlation matrices and their determinants, particularly under elliptical distribution assumptions. By applying the delta method to the Shapley value formula, the authors derive the asymptotic joint distribution of these values. This theoretical foundation then allows for the construction of confidence intervals and hypothesis tests that are computationally superior to bootstrap methods. The paper also details how to estimate the necessary asymptotic covariance terms using sample statistics, making the method practical for real-world applications.
Datasets
The paper uses simulated data for Monte Carlo studies to evaluate the performance of their proposed method. They also apply their method to a real estate dataset from Melbourne, Australia, to investigate changes in property price attribution before and during the early stages of the COVID-19 pandemic.
Key Insights and Ideas
- Asymptotic Normality is Key: The paper's main theoretical contribution is establishing the asymptotic normality of Shapley values for R², paving the way for analytical confidence interval construction.
- Computational Efficiency: The derived asymptotic method is significantly faster than traditional bootstrap approaches, making it feasible for larger datasets or more complex models.
- Reliable Coverage: Monte Carlo simulations demonstrate that the proposed confidence intervals achieve good coverage probabilities, often outperforming bootstrap intervals, especially for smaller sample sizes and certain correlation structures.
- Real-World Application: The application to Melbourne real estate data highlights how Shapley value confidence intervals can reveal significant shifts in predictor importance, even when overall R² values remain similar, particularly in the context of the COVID-19 pandemic.
Key Results
- The proposed asymptotic confidence intervals are computationally superior to bootstrap methods, with execution times significantly lower for the tested scenarios.
- Monte Carlo studies show that the asymptotic confidence intervals achieve coverage probabilities of at least 0.85 across various simulation settings, often outperforming bootstrap CIs for small sample sizes (n < 50).
- In the Melbourne real estate application, the method identified significant differences in the attribution of house prices to features like "distance to CBD" between 2019 and 2020, suggesting a shift in market dynamics possibly influenced by the pandemic.
Limitations and Future Work
- Elliptical Distribution Assumption: The theoretical derivation relies on elliptical or pseudo-elliptical distribution assumptions for the data. While this class is broad, it might not cover all possible data distributions.
- Computational Complexity: While more efficient than bootstrap, the complexity of calculating the asymptotic covariance terms still grows with the number of features (d).
- Future Work: The authors plan to extend their work to derive asymptotic distributions for variance inflation factors and explore Owen values decomposition. They also aim to release R, Julia, and Python implementations of their methods.
Summary updated