Right for Better Reasons: Training Differentiable Models by Constraining their Influence Functions

Xiaoting Shao, Arseny Skryagin, Wolfgang Stammer, Patrick Schramowski, Kristian Kersting

Introduction

Deep learning models often achieve high accuracy by latching onto spurious correlations—a phenomenon known as the "Clever Hans" effect. For example, a model might learn to identify "horses" by looking at a copyright tag in the corner of an image rather than the animal itself. This paper introduces Right for Better Reasons (RBR), a training method that uses influence functions to constrain a model's decision-making process, ensuring it focuses on relevant features rather than "cheating" with background artifacts. Compared to previous gradient-based methods, RBR offers better adversarial robustness, faster convergence, and more interpretable explanations.

The Problem with Input Gradients

Modern neural networks are remarkably good at finding patterns, but they aren't always the patterns we want. Previous attempts to fix this, such as the "Right for the Right Reasons" (RRR) framework, use input gradients to penalize the model when it focuses on the wrong pixels. However, input gradients are often noisy and only represent a local approximation of the decision boundary. They tell us how a prediction changes if we nudge a pixel, but they don't capture how the entire learning process is influenced by that data point.

Methodology: Influence Functions

The authors propose RBR, which replaces simple input gradients with influence functions—a classic tool from robust statistics. While an input gradient looks at a fixed model, an influence function asks: "How would the model's parameters change if we slightly perturbed this training point?" By tracing predictions back through the learning algorithm to the training data, RBR provides a much richer and more robust signal for what the model finds important.

As illustrated in the comparison below, the influence-based explanation (IF) is significantly sharper and less noisy than standard input gradients (IG), providing a clearer map of the decision boundary.

Influence Function (IF) explanation showing sharp feature focus.

The core of RBR is a modified loss function that combines standard cross-entropy with a "Right Reason Loss"—a penalty term using the product of influence functions and input gradients, masked by user-provided feedback (e.g., a mask saying "ignore the bottom-left corner").

Key Results and Insights

  • Adversarial Robustness: By constraining the influence function, the model becomes naturally more resistant to adversarial attacks. In tests using the Fast Gradient Sign Method (FGSM), RBR maintained significantly higher accuracy as perturbation levels increased, while vanilla and RRR models' performance plummeted.
  • Faster Correction: Because the RBR signal is more robust, the model "learns its lesson" faster. In experiments on decoyed data, RBR converged to the correct logic in fewer epochs than previous methods.
  • Real-World Impact: On the PASCAL VOC benchmark, RBR improved accuracy on "counter examples" (images where spurious source tags were removed) to 84%, compared to 71% for a vanilla model and 81% for RRR.
  • Scientific Accuracy: In plant phenotyping tasks, RBR successfully shifted the model's focus from the background to biologically plausible features on the leaves.

Limitations and Future Work

The primary challenge for RBR is computational complexity. Calculating influence functions requires inverting the Hessian matrix, which is expensive for large neural networks. While the authors use efficient approximations, scaling this to massive architectures remains a hurdle. Future work includes moving toward human-in-the-loop interaction and developing ways for users to provide more nuanced instructions beyond simple binary masks.

Summary updated

Abstract

Explaining black-box models such as deep neural networks is becoming increasingly important as it helps to boost trust and debugging. Popular forms of explanations map the features to a vector indicating their individual importance to a decision on the instance-level. They can then be used to prevent the model from learning the wrong bias in data possibly due to ambiguity. For instance, Ross et al.'s ``right for the right reasons'' propagates user explanations backwards to the network by formulating differentiable constraints based on input gradients. Unfortunately, input gradients as well as many other widely used explanation methods form an approximation of the decision boundary and assume the underlying model to be fixed. Here, we demonstrate how to make use of influence functions---a well known robust statistic---in the constraints to correct the model’s behaviour more effectively. Our empirical evidence demonstrates that this ``right for better reasons''(RBR) considerably reduces the time to correct the classifier at training time and boosts the quality of explanations at inference time compared to input gradients. Besides, we also showcase the effectiveness of RBR in correcting "Clever Hans"-like behaviour in real, high-dimensional domain.
BibTeX
@inproceedings{shao2021right,
  author = {Xiaoting Shao and Arseny Skryagin and Wolfgang Stammer and Patrick Schramowski and Kristian Kersting},
  title = {{Right for Better Reasons: Training Differentiable Models by Constraining their Influence Functions}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2021},
  doi = {10.1609/aaai.v35i11.17148},
}
Figure