Introduction
Deep learning models are frequently criticized as "black boxes," making interpretation methods essential for establishing trust and safety. These methods aim to identify the internal neurons or filters responsible for specific semantic concepts, such as "wheels" or "fur." However, state-of-the-art interpretation techniques often impose a significant "computational tax," requiring the processing of entire massive datasets (like ImageNet) to extract stable and reliable insights. This paper addresses this data-hungry nature by proposing a framework that achieves high-quality model transparency using only a small fraction of the original data.
The Coreset-Based Framework
The authors introduce a framework centered on coresets—representative data subsets that capture the geometric and statistical essence of the full dataset. The process follows a two-step pipeline:
- Coreset Selection: Geometry-based methods are used to select a subset of data. The study evaluates Moderate Selection (samples near the median of the activation space) and DGPruning (a diversity-graph-based approach) against simple random sampling.
- Interpretation: Three popular post-hoc interpretation methods—VEBI, ICE, and Topic-based interpretation—are applied to these subsets to identify relevant internal units.
To validate the effectiveness of these subsets, the authors utilize a similarity-based evaluation protocol. By calculating the "angular distance" between features identified using the full dataset versus those identified using the coreset, they can quantitatively measure how much of the model's logic is preserved.
Key Insights and Efficiency
The study reveals several critical findings regarding the redundancy of data in model explanations:
- Robustness to Data Reduction: Model interpretation is surprisingly resilient. High-quality insights can be achieved with as little as 5-20% of the original training data. At the 20% threshold, representation similarity was found to be nearly identical to using 95% of the dataset.
- Coreset Transferability: A significant discovery is that coresets are transferable across architectures. A data subset selected to interpret a ResNet50 model can be effectively reused to interpret a VGG19 model with minimal loss in explanation quality, eliminating the need to re-run expensive selection algorithms for every new model.
- Method Sensitivity: Not all interpretation methods scale the same way. VEBI proved exceptionally robust at very low sampling rates, while methods like ICE showed slight performance dips as data increased, likely due to the scaling limitations of its underlying Non-negative Matrix Factorization (NMF) components.
Visual and Quantitative Results
The framework's success is most evident in the visual consistency of its outputs. Saliency maps generated from a mere 5% coreset are often visually indistinguishable from those produced using the complete dataset. This suggests that the core "logic" the model uses to identify objects is concentrated in a relatively small number of representative samples.
Furthermore, the authors confirmed the functional relevance of these coreset-identified features. When the important features identified via coresets were "turned off," the model's classification accuracy dropped significantly, matching the degradation seen when using the full dataset for feature identification. This confirms that coresets do not just produce similar visuals, but correctly identify the features the model relies on for decision-making.
Conclusion and Future Directions
By leveraging coresets, researchers can significantly reduce the computational resources required for deep model interpretation without sacrificing the quality of the explanation. While currently focused on Convolutional Neural Networks (CNNs) and visual tasks, this approach paves the way for more efficient transparency in larger architectures like Vision Transformers (ViTs) and Large Language Models (LLMs). Future work may also explore model-agnostic coresets to further streamline the initial selection process.
Summary updated