Abstract
Most convolutional neural network (CNN) channel pruning methods treat the network as a black box, removing filters based solely on their impact on final output error. This paper introduces "White-Box" pruning, which leverages CNN interpretability to evaluate the contribution of internal channels to specific image categories. By training class-wise masks and preserving channels that benefit the most categories, the method achieves state-of-the-art computation (FLOP) reductions with minimal to no accuracy loss. Experimental results on CIFAR-10 and ImageNet demonstrate that this approach consistently outperforms existing pruning strategies.
1. Introduction
Convolutional Neural Networks (CNNs) are computationally intensive, making them difficult to deploy on resource-constrained edge devices. Channel pruning addresses this by removing entire filters to save compute. However, existing pruning methods—whether based on weight magnitude, sparsity constraints, or AutoML—are fundamentally "black-box" approaches. They optimize for the final classification output while ignoring the internal semantic logic of the network.
Visualization tools reveal that specific channels learn to extract highly specific local features (e.g., one channel might activate for animal heads, while another activates for text). The authors hypothesize that channels useful for recognizing many categories are universally important and should be kept, while channels that only help identify a tiny subset of categories are redundant. This paper proposes a "White-Box" pruning strategy that explicitly measures channel importance through category-driven interpretability.
2. Methodology
To open the black box, the authors propose a pipeline that explicitly measures each channel's contribution to every individual class.
2.1 Class-wise Masks and Soft Labels
The authors introduce a trainable mask for each channel, with a length equal to the number of categories in the dataset. During training, when an image of a specific class is processed, only the corresponding index of the mask is activated to scale the channel's output feature map. To prevent overfitting caused by forcing intermediate layers to predict strict one-hot labels, the authors use a "soft label" approach, smoothing the label distributions using a normal distribution for non-target classes.
2.2 Sparsity Regularization and Global Voting
An $L_2$-norm penalty is applied to the masks to encourage sparsity. After a few epochs of training, the absolute sum of a channel's mask values across all classes becomes its "importance score." Channels are ranked globally by this score, and the lowest-scoring channels are iteratively removed until a target FLOP budget is met. Finally, the pruned network is fine-tuned to recover accuracy.
3. Experiments and Results
The White-Box approach was evaluated on CIFAR-10 (using VGG-16, ResNet-56/110, and MobileNet-v2) and ILSVRC-2012 (ImageNet) using ResNet-50.
3.1 Performance Benchmarks
The method consistently outperformed state-of-the-art pruning methods like HRank, GAL, and LFPC. For ResNet-110, it reduced FLOPs by 66.0% while improving top-1 accuracy by 0.62%. For VGG-16, it achieved a 76.4% FLOP reduction with only a negligible 0.45% drop in accuracy.
On ResNet-50, the method achieved a 45.6% FLOP reduction with a minor 0.83% drop in top-1 accuracy. Pushing the pruning further to a 63.5% FLOP reduction resulted in a 1.94% accuracy drop, translating to a 1.62x actual inference speedup on a V100 GPU.
4. Discussion and Conclusion
This work successfully bridges CNN interpretability with practical network compression. By defining importance through universal applicability across classes, the White-Box approach achieves superior compression-accuracy trade-offs. However, the method is highly sensitive to the soft-label smoothing trick, and its reliance on ground-truth labels limits its current application to supervised classification. Future work could adapt this logic to modern architectures like Vision Transformers or multi-label tasks like object detection.
Summary updated