The Directed Prediction Change - Efficient and Trustworthy Fidelity Assessment for Local Feature Attribution Methods

Kevin Iselborn, David Dembinsky, Adriano Lucieri, Andreas DengelarXiv 2025 · DOI · arXiv

Introduction: The Fidelity Gap in Feature Attribution

As artificial intelligence is increasingly deployed in high-stakes domains such as medicine and finance, the demand for interpretability has grown. Feature Attribution (FA) methods, like Saliency Maps, aim to explain model decisions by highlighting influential input features. However, a critical question remains: how faithful are these explanations to the model's actual internal logic? This is the problem of fidelity.

Currently, researchers rely on two primary metrics to assess fidelity, both of which possess significant drawbacks. Infidelity is considered highly accurate but depends on Monte Carlo sampling, making it computationally expensive and stochastic. As shown in Figure 1, Infidelity requires a high number of samples to converge, and its inherent randomness can lead to inconsistent results across evaluations.

Infidelity metric convergence and variance across Monte-Carlo samples.

Conversely, Prediction Change (PC) is fast and deterministic but suffers from a fundamental bias. While it effectively evaluates "baseline-oriented" explanations (comparing inputs to a reference like a black square), it often fails to recognize the validity of "local" explanations (how the model behaves in the immediate neighborhood of a data point), frequently ranking them no better than random noise.

Methodology: Directed Prediction Change (DPC)

To bridge this gap, researchers have introduced Directed Prediction Change (DPC). DPC refines the standard Guided Perturbation Experiment—where features are systematically removed to observe changes in model output—by incorporating directionality.

Standard PC metrics focus solely on the magnitude of change in the model's prediction. DPC, however, incorporates the mathematical sign (positive or negative) of both the attribution and the perturbation. The core logic is intuitive: if an explanation identifies a feature as supporting a prediction, a decrease in that feature should logically lead to a decrease in the model's score. By penalizing misalignments in direction, DPC captures the same underlying fidelity property as Infidelity but does so through a deterministic, non-sampling-based process.

Experimental Evaluation

The robustness of DPC was tested across two distinct data modalities and model architectures:

  • HELOC (Home Equity Line of Credit): A tabular dataset for financial risk assessment, evaluated using Linear models and Multi-Layer Perceptrons (MLPs).
  • ISIC (International Skin Imaging Collaboration): A large-scale image dataset for skin lesion classification, evaluated using Linear models and InceptionV1 architectures.

These experiments compared DPC against a suite of attribution methods, categorized into "Baseline Oriented" (e.g., DeepLiftSHAP, Integrated Gradients) and "Local" (e.g., Grad, SmoothGrad).

Key Results and Discussion

The evaluation reveals that DPC successfully addresses the limitations of existing metrics:

  1. Massive Speedup: On an A100 GPU, DPC achieved a median speedup of 9.91x compared to Infidelity, significantly reducing the computational overhead for model validation.
  2. Correcting the Bias: Standard PC metrics systematically undervalue local explanations. As seen in Figure 4, local methods like "Grad" often show near-zero fidelity in standard PC benchmarks, even in linear models where they are mathematically optimal. DPC corrects this bias, accurately identifying Gradient as a top-performing method in linear contexts.
  3. Trustworthy and Deterministic: By eliminating Monte Carlo sampling, DPC provides a reproducible benchmark. This reproducibility is essential for regulated industries where "randomness" in evaluation is unacceptable.
Fidelity assessment across different models and datasets using standard Prediction Change (PC).

Furthermore, DPC showed a strong anti-correlation with Infidelity validation scores, confirming that it measures the same fundamental fidelity property while offering superior performance and stability.

Limitations and Future Work

The authors note that DPC’s performance is slightly less pronounced on complex image data compared to tabular data, likely due to conflicting directions among many pixels canceling each other out. Future research will focus on mitigating "out-of-distribution" issues common in perturbation-based metrics. By employing more "local" perturbations—small nudges rather than total feature removal—the researchers aim to further refine the accuracy of DPC across all data modalities.

Summary updated

Abstract

The utility of an explanation method critically depends on its fidelity to the underlying machine learning model. Especially in high-stakes medical settings, clinicians and regulators require explanations that faithfully reflect the model's decision process. Existing fidelity metrics such as Infidelity rely on Monte Carlo approximation, which demands numerous model evaluations and introduces uncertainty due to random sampling. This work proposes a novel metric for evaluating the fidelity of local feature attribution methods by modifying the existing Prediction Change (PC) metric within the Guided Perturbation Experiment. By incorporating the direction of both perturbation and attribution, the proposed Directed Prediction Change (DPC) metric achieves an almost tenfold speedup and eliminates randomness, resulting in a deterministic and trustworthy evaluation procedure that measures the same property as local Infidelity. DPC is evaluated on two datasets (skin lesion images and financial tabular data), two black-box models, seven explanation algorithms, and a wide range of hyperparameters. Across $4\,744$ distinct explanations, the results demonstrate that DPC, together with PC, enables a holistic and computationally efficient evaluation of both baseline-oriented and local feature attribution methods, while providing deterministic and reproducible outcomes.
BibTeX
@misc{iselborn2025directed,
  author = {Kevin Iselborn and David Dembinsky and Adriano Lucieri and Andreas Dengel},
  title = {{The Directed Prediction Change - Efficient and Trustworthy Fidelity Assessment for Local Feature Attribution Methods}},
  year = {2025},
  eprint = {2511.21363},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2511.21363},
}
Figure