Structured Pruning for Efficient ConvNets via Incremental Regularization

Huan Wang, Qiming Zhang, Yuehai Wang, Lu Yu, Haoji Hu

Abstract

Parameter pruning is a promising approach for CNN compression and acceleration by eliminating redundant model parameters with tolerable performance degrade. Despite its effectiveness, existing regularization-based parameter pruning methods usually drive weights towards zero with large and constant regularization factors, which neglects the fragility of the expressiveness of CNNs, and thus calls for a more gentle regularization scheme so that the networks can adapt during pruning. To achieve this, we propose a new and novel regularization-based pruning method, named IncReg, to incrementally assign different regularization factors to different weights based on their relative importance. Empirical analysis on CIFAR-10 dataset verifies the merits of IncReg. Further extensive experiments with popular CNNs on CIFAR-10 and ImageNet datasets show that IncReg achieves comparable to even better results compared with state-of-the-arts. Our source codes and trained models are available here: https://github.com/mingsun-tse/caffe_increg.
BibTeX
@inproceedings{wang2019structured,
  author = {Huan Wang and Qiming Zhang and Yuehai Wang and Lu Yu and Haoji Hu},
  title = {{Structured Pruning for Efficient ConvNets via Incremental Regularization}},
  booktitle = {IJCNN},
  year = {2019},
  doi = {10.1109/ijcnn.2019.8852463},
}

Loading related content...

Figure