Introduction: The Fidelity Gap in Feature Attribution
As artificial intelligence is increasingly deployed in high-stakes domains such as medicine and finance, the demand for interpretability has grown. Feature Attribution (FA) methods, like Saliency Maps, aim to explain model decisions by highlighting influential input features. However, a critical question remains: how faithful are these explanations to the model's actual internal logic? This is the problem of fidelity.
Currently, researchers rely on two primary metrics to assess fidelity, both of which possess significant drawbacks. Infidelity is considered highly accurate but depends on Monte Carlo sampling, making it computationally expensive and stochastic. As shown in Figure 1, Infidelity requires a high number of samples to converge, and its inherent randomness can lead to inconsistent results across evaluations.
Conversely, Prediction Change (PC) is fast and deterministic but suffers from a fundamental bias. While it effectively evaluates "baseline-oriented" explanations (comparing inputs to a reference like a black square), it often fails to recognize the validity of "local" explanations (how the model behaves in the immediate neighborhood of a data point), frequently ranking them no better than random noise.
Methodology: Directed Prediction Change (DPC)
To bridge this gap, researchers have introduced Directed Prediction Change (DPC). DPC refines the standard Guided Perturbation Experiment—where features are systematically removed to observe changes in model output—by incorporating directionality.
Standard PC metrics focus solely on the magnitude of change in the model's prediction. DPC, however, incorporates the mathematical sign (positive or negative) of both the attribution and the perturbation. The core logic is intuitive: if an explanation identifies a feature as supporting a prediction, a decrease in that feature should logically lead to a decrease in the model's score. By penalizing misalignments in direction, DPC captures the same underlying fidelity property as Infidelity but does so through a deterministic, non-sampling-based process.
Experimental Evaluation
The robustness of DPC was tested across two distinct data modalities and model architectures:
- HELOC (Home Equity Line of Credit): A tabular dataset for financial risk assessment, evaluated using Linear models and Multi-Layer Perceptrons (MLPs).
- ISIC (International Skin Imaging Collaboration): A large-scale image dataset for skin lesion classification, evaluated using Linear models and InceptionV1 architectures.
These experiments compared DPC against a suite of attribution methods, categorized into "Baseline Oriented" (e.g., DeepLiftSHAP, Integrated Gradients) and "Local" (e.g., Grad, SmoothGrad).
Key Results and Discussion
The evaluation reveals that DPC successfully addresses the limitations of existing metrics:
- Massive Speedup: On an A100 GPU, DPC achieved a median speedup of 9.91x compared to Infidelity, significantly reducing the computational overhead for model validation.
- Correcting the Bias: Standard PC metrics systematically undervalue local explanations. As seen in Figure 4, local methods like "Grad" often show near-zero fidelity in standard PC benchmarks, even in linear models where they are mathematically optimal. DPC corrects this bias, accurately identifying Gradient as a top-performing method in linear contexts.
- Trustworthy and Deterministic: By eliminating Monte Carlo sampling, DPC provides a reproducible benchmark. This reproducibility is essential for regulated industries where "randomness" in evaluation is unacceptable.
Furthermore, DPC showed a strong anti-correlation with Infidelity validation scores, confirming that it measures the same fundamental fidelity property while offering superior performance and stability.
Limitations and Future Work
The authors note that DPC’s performance is slightly less pronounced on complex image data compared to tabular data, likely due to conflicting directions among many pixels canceling each other out. Future research will focus on mitigating "out-of-distribution" issues common in perturbation-based metrics. By employing more "local" perturbations—small nudges rather than total feature removal—the researchers aim to further refine the accuracy of DPC across all data modalities.
Summary updated