Abstract
Modern deep learning faces a persistent trade-off between the high performance of "black-box" models and the transparency of "white-box" systems. While post-hoc explanation methods like Shapley values offer a path toward interpretability, they are often computationally prohibitive or require significant model modification. This paper introduces AutoGnothi, a framework that leverages parameter-efficient side-tuning to transform Transformers into self-interpretable systems. By generating theoretically grounded explanations in a single inference pass, AutoGnothi reduces computational overhead by over 50% without sacrificing predictive accuracy.
The Interpretability-Efficiency Trade-off
In Explainable AI (XAI), researchers typically choose between two suboptimal paths. White-box models (e.g., decision trees) are inherently interpretable but lack the capacity for complex tasks. Conversely, black-box models (e.g., Transformers) provide state-of-the-art performance but remain opaque.
To bridge this gap, practitioners often use post-hoc explainers. The gold standard, Shapley values, attributes a prediction to input features based on game theory. However, calculating exact Shapley values is computationally expensive, often requiring thousands of model evaluations per instance. Even modern proxy explainers require training massive auxiliary networks, leading to significant memory and deployment challenges. Furthermore, modifying a model to make it explainable raises a philosophical concern akin to Theseus’s Paradox: if the model is significantly altered, is the explanation still faithful to the original logic?
Methodology: AutoGnothi and Ladder Side-Tuning
AutoGnothi (“Know Thyself”) addresses these challenges by integrating interpretability directly into the model architecture using Parameter-Efficient Transfer Learning (PETL).
Instead of full fine-tuning, the framework employs Ladder Side-Tuning (LST). In this setup, the original backbone model (the "black box") remains frozen, preserving its original logic and avoiding Theseus’s Paradox. A slim auxiliary network—the "side ladder"—is constructed to tap into the features of the backbone at every layer.
The training process occurs in two distinct stages:
- Surrogate Stage: A small side network is trained to mimic the backbone's behavior, specifically learning to predict accurately even when input features are masked. This teaches the system the relative importance of different features.
- Explainer Stage: A second side network is trained to estimate Shapley values based on the surrogate’s learned feature contributions.
At inference time, the model produces both the prediction and the explanation simultaneously in a single forward pass, drastically reducing the time required for post-hoc analysis.
Experimental Validation
The researchers evaluated AutoGnothi across diverse domains and architectures:
- Vision: ImageNette, Oxford-IIIT Pets, and MURA (medical imaging) using Vision Transformers (ViT-Tiny to ViT-Large).
- Language: Yelp Review Polarity for sentiment analysis using BERT-Base.
Key Insights
- Intrinsic Correlation: The authors found a high correlation between features used for prediction and those needed for explanation. By leveraging the frozen backbone, the side network avoids "relearning" the world, focusing instead on formatting existing knowledge into interpretable outputs.
- Theoretical Convergence: Unlike many heuristic saliency maps, AutoGnothi is supported by mathematical proofs guaranteeing that the side-tuned explainer converges toward true Shapley values.
Results and Impact
AutoGnothi effectively eliminates the traditional trade-off between efficiency and interpretability:
- Parameter Efficiency: For a ViT-Base model, AutoGnothi reduced trainable parameters by 97-98% compared to standard fine-tuning methods.
- Resource Savings: The framework slashed training memory usage by 72-77% and reduced inference time by 44%.
- Explanation Quality: In "insertion/deletion" benchmarks—which measure how well an explanation tracks with a model's actual logic—AutoGnothi consistently outperformed 12 state-of-the-art methods, including GradCAM and RISE. The resulting heatmaps were significantly cleaner, focusing on relevant objects rather than background noise.
Conclusion and Future Work
AutoGnothi demonstrates that self-interpretability can be achieved with minimal overhead. While this work focused on Transformer architectures, the underlying logic of side-tuning is theoretically applicable to other models like CNNs or Mamba. Future research will explore the scalability of this approach to large-scale frontier models exceeding 100 billion parameters, where the efficiency gains of side-tuning would be even more critical.
Summary updated