Introduction
In high-stakes fields like healthcare and finance, understanding why a machine learning model makes a specific decision is critical. Currently, practitioners rely heavily on "local attribution" methods like LIME or Local Shapley Values (L-SV) to explain individual predictions.
However, these popular tools harbor a fundamental flaw: they often lack "soundness." A sound attribution method should only assign importance to features that genuinely influence the model's output or are statistically dependent on features that do. Recent analysis reveals that LIME and SHAP frequently fail this test, particularly with piecewise linear models—the mathematical backbone of modern neural networks using ReLU activations. Because these methods rely on local approximations or averaging across many subsets, they often "bleed" global influences into local explanations. Consequently, a feature might appear important for a specific prediction even when the model's logic for that exact instance ignores it entirely.
To address this, researchers introduce R-LOCO (Regional Leave Out COvariates), a novel framework that clusters data points based on how the model behaves toward them, providing "regional" explanations that are more accurate, stable, and computationally efficient than traditional local methods.
The R-LOCO Framework
R-LOCO bridges the gap between local and global explanations through a three-step methodology:
- Importance Representation: Instead of analyzing raw input data, R-LOCO transforms each data point into a "feature importance space." It utilizes a global metric called LOCO (Leave Out COvariates), which measures the drop in model performance when a specific feature is removed.
- Regional Clustering: Observations exhibiting similar importance profiles are grouped together. Crucially, this clustering occurs in the importance space rather than the input space. This ensures points are grouped because the model "treats" them similarly functionally, rather than just because their raw feature values are proximate.
- Regional Attribution: For any new instance, the system identifies its corresponding cluster (region) and assigns it the average importance of that region.
By defining "local" through functional similarity rather than physical proximity in the input space, R-LOCO filters out noise from irrelevant features and captures instance-specific details while maintaining the statistical rigor of global methods.
Experimental Validation
The researchers evaluated R-LOCO using both controlled synthetic environments (1st-order, 2nd-order, and interaction models where the "ground truth" of feature importance is known) and real-world benchmarks (Diabetes and California Housing datasets using XGBoost).
Key Findings
- Higher Fidelity and Accuracy: In synthetic tests, R-LOCO achieved a True Positive rate of ~97% in identifying important features, drastically outperforming Local Shapley Values (41%) and LIME (53%).
- Fewer False Positives: R-LOCO assigned minimal importance (1-4%) to irrelevant features. In contrast, LIME and SHAP assigned significantly higher "ghost" importance (7-18%) to features that played no role in the prediction.
- Inference Speed: Because R-LOCO reduces the explanation task to a simple cluster assignment at test time, it is orders of magnitude faster. On the California Housing dataset, R-LOCO required only 0.032 seconds for inference, compared to 34.6 seconds for LIME and 20.6 seconds for SHAP.
To further validate the feature rankings, the researchers measured performance drops when perturbing the most and least important features. As shown below, R-LOCO consistently demonstrates superior feature ranking compared to baseline methods.
Limitations and Future Directions
While R-LOCO offers significant improvements, it relies heavily on the quality of the underlying clustering algorithm. If regions are "contaminated" by mixing points from different functional parts of the model, the resulting attributions lose accuracy. Additionally, R-LOCO implicitly assumes the model can be approximated by an additive structure within each region.
Future work will explore enriching the importance representation by incorporating second-order or higher-order feature interactions, allowing the framework to better capture the complex dependencies inherent in real-world data.
Summary updated