Abstract
Local Interpretable Model-agnostic Explanations (LIME) is a widely used framework for explaining the predictions of complex black-box models. However, standard LIME implementations often produce "confused" or noisy heatmaps in image classification tasks. This paper identifies the root cause as the limitations of Monte Carlo (MC) sampling in high-dimensional feature spaces. We propose a stratified sampling approach that ensures the synthetic neighborhood captures the full range of model behavior. By applying a mathematical adjustment factor to maintain unbiased estimation, our method produces significantly more stable, precise, and visually clear feature attributions.
Introduction
LIME explains individual predictions by perturbing the input and observing the resulting changes in the model's output. For images, this involves segmenting the image into superpixels and randomly masking them to create a "synthetic neighborhood." A simple linear model is then fitted to these samples to weight the importance of each superpixel.
Despite its popularity, LIME often suffers from low-contrast explanations where importance scores are nearly uniform. This paper argues that the default Monte Carlo sampling strategy fails to provide the linear regressor with a sufficiently diverse set of data points to find a meaningful gradient, especially as the number of superpixels increases.
The Sampling Problem in High Dimensions
In standard LIME, each superpixel has a 50% probability of being masked. In high-dimensional spaces (e.g., images with 50+ superpixels), the binomial distribution of masked pixels becomes highly concentrated. Most generated samples will have approximately half of their pixels masked, creating a "hypersphere" of samples at a uniform distance from the original image.
This lack of diversity means the linear model rarely sees samples where only a few features are changed or where almost all features are removed. Consequently, the range of the model's output is squashed, leading to a "confused" regressor that cannot distinguish between relevant and irrelevant features.
Proposed Method: Stratified Sampling
To address this, we replace Monte Carlo sampling with a stratified approach designed to cover the entire spectrum of feature presence.
Stratification Strategy
Instead of independent coin flips for each superpixel, we first select a "stratum" $s$, representing the number of superpixels to preserve (ranging from 0 to the total number $k$). We then sample uniformly within these strata. This ensures that the neighborhood includes rare but informative cases, such as images with nearly all pixels masked or nearly none.
Unbiased Estimation
Because stratified sampling intentionally over-represents rare cases, it introduces a sampling bias. We derive a mathematical adjustment factor ($adj$) to weight samples during the linear regression process. This correction ensures that the resulting explanation remains an unbiased estimator of the local model behavior while benefiting from the increased variance in the input space.
Experimental Evaluation
We evaluated the proposed method using the ImageNet Object Localization dataset and a pre-trained ResNet50 model. We compared standard MC-LIME against our Stratified-LIME across several metrics:
- Range Coverage (RC): Measures how well the sampled neighborhood covers the model's output spectrum (0 to the original prediction). Stratified sampling consistently achieved $RC \approx 1.0$, while MC sampling performance degraded as the number of superpixels increased.
- Coefficient of Variation (CV): Measures the contrast in feature weights. Stratified sampling maintained high CV, indicating sharp, localized explanations.
- Goodness of Fit ($R^2$): Stratified sampling resulted in significantly higher $R^2$ values, suggesting the surrogate linear model was a more faithful local representation of the ResNet50 model.
Results and Discussion
Qualitative results demonstrate that stratified sampling produces sharp heatmaps that clearly identify diagnostic features (e.g., the specific markings of a bird or animal). In contrast, MC sampling often produced blurry, red-tinted blobs that failed to localize the model's focus. Quantitatively, the stability of stratified sampling remained robust even as image segmentation became finer (increasing $k$ from 50 to 200), whereas MC-LIME reliability plummeted.
Furthermore, we highlight a theoretical connection to Shapley values, showing that LIME's default sampling is mathematically the reciprocal of what Shapley theory suggests is optimal for feature attribution, providing further theoretical justification for our stratified approach.
Conclusion
Stratified sampling effectively solves the "confusion" problem in LIME image explanations by ensuring the synthetic neighborhood is well-represented across all levels of feature presence. This approach provides more faithful, stable, and interpretable insights into deep learning models. Future work will explore extending this stratification framework to tabular and text data, as well as investigating interactions with different regularization techniques.
Summary updated