Introduction
Deep learning models often achieve high accuracy by latching onto spurious correlations—a phenomenon known as the "Clever Hans" effect. For example, a model might learn to identify "horses" by looking at a copyright tag in the corner of an image rather than the animal itself. This paper introduces Right for Better Reasons (RBR), a training method that uses influence functions to constrain a model's decision-making process, ensuring it focuses on relevant features rather than "cheating" with background artifacts. Compared to previous gradient-based methods, RBR offers better adversarial robustness, faster convergence, and more interpretable explanations.
The Problem with Input Gradients
Modern neural networks are remarkably good at finding patterns, but they aren't always the patterns we want. Previous attempts to fix this, such as the "Right for the Right Reasons" (RRR) framework, use input gradients to penalize the model when it focuses on the wrong pixels. However, input gradients are often noisy and only represent a local approximation of the decision boundary. They tell us how a prediction changes if we nudge a pixel, but they don't capture how the entire learning process is influenced by that data point.
Methodology: Influence Functions
The authors propose RBR, which replaces simple input gradients with influence functions—a classic tool from robust statistics. While an input gradient looks at a fixed model, an influence function asks: "How would the model's parameters change if we slightly perturbed this training point?" By tracing predictions back through the learning algorithm to the training data, RBR provides a much richer and more robust signal for what the model finds important.
As illustrated in the comparison below, the influence-based explanation (IF) is significantly sharper and less noisy than standard input gradients (IG), providing a clearer map of the decision boundary.
The core of RBR is a modified loss function that combines standard cross-entropy with a "Right Reason Loss"—a penalty term using the product of influence functions and input gradients, masked by user-provided feedback (e.g., a mask saying "ignore the bottom-left corner").
Key Results and Insights
- Adversarial Robustness: By constraining the influence function, the model becomes naturally more resistant to adversarial attacks. In tests using the Fast Gradient Sign Method (FGSM), RBR maintained significantly higher accuracy as perturbation levels increased, while vanilla and RRR models' performance plummeted.
- Faster Correction: Because the RBR signal is more robust, the model "learns its lesson" faster. In experiments on decoyed data, RBR converged to the correct logic in fewer epochs than previous methods.
- Real-World Impact: On the PASCAL VOC benchmark, RBR improved accuracy on "counter examples" (images where spurious source tags were removed) to 84%, compared to 71% for a vanilla model and 81% for RRR.
- Scientific Accuracy: In plant phenotyping tasks, RBR successfully shifted the model's focus from the background to biologically plausible features on the leaves.
Limitations and Future Work
The primary challenge for RBR is computational complexity. Calculating influence functions requires inverting the Hessian matrix, which is expensive for large neural networks. While the authors use efficient approximations, scaling this to massive architectures remains a hurdle. Future work includes moving toward human-in-the-loop interaction and developing ways for users to provide more nuanced instructions beyond simple binary masks.
Summary updated