Introduction
In the quest for "Explainable AI," one powerful technique is Representer Point Selection (RPS). Instead of generating a heatmap of pixels (saliency maps), RPS identifies the specific training examples that most influenced a model's decision. It answers the question: "Why did the model classify this image as a car? Because it learned from these specific training images."
However, the existing state-of-the-art method, RPS-$l_2$, suffers from two major flaws:
- Model Mismatch: It requires fine-tuning the model with $l_2$ regularization to work. Consequently, the explanation provided is for a modified version of the model, not the original one deployed in production.
- The "Generic" Problem: It often produces nearly identical explanations for every test point in the same class. If you ask why it classified ten different images as "dogs," it might point to the exact same three training images every time, failing to provide instance-specific insight.
To address these limitations, researchers have developed RPS-LJE (Local Jacobian Expansion), a new method that provides individualized and stable explanations without requiring the model to be retrained or modified.
Methods: Local Jacobian Expansion
Unlike previous methods that rely on optimization tricks like $l_2$ regularization, RPS-LJE uses a first-order Taylor expansion on the Jacobian matrix of the model's loss function.
- Post-hoc Application: It works directly on the already-trained model. No fine-tuning or retraining is required.
- Theoretical Foundation: It leverages the Representer Theorem to decompose a prediction into a weighted sum of training points. By incorporating second-order information (the Hessian matrix), it accounts for the correlations between model parameters.
- Broad Compatibility: The method requires the model to have a linear last layer. While this is a standard feature in ResNets and Transformers, RPS-LJE is also remarkably flexible, extending even to gradient-boosted trees like XGBoost.
Datasets and Evaluation
The authors evaluated RPS-LJE across three distinct domains to demonstrate its versatility:
- Images (CIFAR-10): A binary "horses vs. cars" classification task using a ResNet-20 architecture.
- Text (IMDB): Sentiment analysis using a Bi-LSTM (Bidirectional Long Short-Term Memory) network.
- Tabular (German Credit): Credit risk identification using XGBoost, proving the method works for ensemble tree models, not just deep neural networks.
Key Insights and Ideas
Breaking the Class-Level Curse
The authors discovered that in the older RPS-$l_2$ method, the first-order derivative of the loss was so dominant that it drowned out the unique features of the test point. By using the Jacobian expansion, RPS-LJE allows the specific features of the test instance to shine through, leading to truly individualized explanations.
Stability and XGBoost Support
RPS-$l_2$ was highly sensitive to the strength of the $l_2$ penalty—changing the hyperparameter could alter the explanation entirely. RPS-LJE is remarkably stable across different settings, making it a more reliable debugging tool. Furthermore, because RPS-LJE doesn't require fine-tuning (which is notoriously difficult for gradient-boosted trees), it opens up sample-based explanations for a massive class of models used in industry that were previously hard to explain.
Key Results
- Data Debugging: In a task designed to find mislabeled training data (data curation), RPS-LJE outperformed both the original RPS and Influence Functions. On the CIFAR-10 task, it was 3% more effective at identifying corrupt labels after checking 40% of the data.
- Qualitative Superiority: Visually, the explanations make much more sense. For a test image of a red truck, RPS-LJE pointed to other red trucks in the training set, whereas the old method often pointed to generic images that appeared in the explanation lists for almost every "car" prediction.
- Coherence in Text: In text analysis, RPS-LJE identified training examples that shared the same narrative style or sentence structure as the test point, providing a much more intuitive "reasoning" for the model's sentiment prediction.
Limitations and Future Work
While powerful, RPS-LJE has a few limitations:
- Linear Last Layer: The method assumes the model has a feature embedding stage followed by a linear prediction layer. While this covers most modern architectures, it isn't universal.
- Saddle Point Assumption: The mathematics assume the model has been trained to near-convergence (a "saddle point" where gradients are near zero). If a model is heavily under-trained or uses extreme early stopping, the Taylor expansion might be less accurate.
- Computational Overhead: Like many methods involving the Hessian (second-order derivatives), calculating the inverse can be computationally expensive for very large models, though the authors utilize efficient approximations.
Future work could involve extending this logic to models without a clear linear last layer or finding ways to further scale the Hessian calculations for massive "foundation" models.
Summary updated