Constraint-Aware Importance Estimation for Global Filter Pruning under Multiple Resource Constraints

Yu-Cheng Wu, Chih-Ting Liu, Bo-Ying Chen, Shao-Yi ChienWorkshops @ CVPR 2020 · DOI

Abstract

Filter pruning is a common technique for compressing deep neural networks, yet standard importance estimation methods often neglect the varying resource costs of individual filters. This paper presents Constraint-Aware Importance Estimation (CAIE), a novel pruning framework that prioritizes filters based on their efficiency—the ratio of accuracy impact to resource consumption. By modeling resource impacts as vectors, CAIE effectively handles multiple simultaneous hardware constraints. Experimental results on ImageNet and CIFAR-10 demonstrate that CAIE achieves superior accuracy-compression trade-offs compared to state-of-the-art methods while precisely meeting target resource budgets.

Introduction

Deep learning models are increasingly deployed on resource-constrained mobile and embedded devices. Filter pruning—the removal of entire convolutional filters—is a primary strategy for reducing model size and computational cost. Traditional pruning methods typically estimate a filter's "importance" based solely on its contribution to the model's accuracy (e.g., loss impact). However, this approach ignores the "cost" of the filter in terms of FLOPs or parameter count. In heterogeneous network architectures, filters with similar accuracy impacts can have vastly different resource footprints. Furthermore, real-world deployment often necessitates satisfying multiple constraints simultaneously, such as a maximum memory footprint and a specific latency target.

Methodology: Constraint-Aware Importance Estimation (CAIE)

The authors propose shifting the pruning criterion from absolute importance to relative efficiency. This "Return on Investment" (ROI) mentality ensures that the pruning process prioritizes removing filters that provide the most resource savings for the least accuracy cost.

Single-Constraint Efficiency

For a single resource constraint (e.g., FLOPs), the CAIE score $I(f)$ for a filter $f$ is defined as the ratio of loss impact $\ell(f)$ to resource impact $r(f)$.

Comparison of pruning trajectories: CAIE vs. standard methods.

Handling Multiple Constraints

When multiple resources are involved, the resource impact becomes a vector $\vec{r}(f)$. CAIE calculates an effective resource impact $r_e(f)$ by projecting the filter's impact vector onto the overall "pruning objective vector" $\vec{R}$, which points toward the target resource state. This allows the algorithm to dynamically balance multiple constraints (e.g., FLOPs and parameters) throughout the iterative pruning process.

The Global Pruning Pipeline

CAIE employs an iterative "Prune-then-Fine-tune" workflow:

  1. Estimation: Rapidly estimate loss impact using first-order Taylor expansion.
  2. Calculation: Determine resource impacts (FLOPs, parameters) for each filter.
  3. Scoring: Compute CAIE scores based on current resource gaps and vector projections.
  4. Iteration: Globally remove the least efficient filters across the entire network.
  5. Fine-tuning: Recover accuracy through a final training phase.

Experimental Results

The CAIE method was evaluated on CIFAR-10 and ImageNet (ILSVRC-12) using ResNet-50, ResNet-34, and VGG16-BN architectures.

Performance and Precision

CAIE consistently outperformed state-of-the-art baselines such as Taylor-FO-BN, Thinet, and FPGM. For instance, on ImageNet with ResNet-50, CAIE achieved a Top-1 accuracy of 72.39% at ~33% FLOPs, surpassing the Taylor-FO-BN baseline of 71.69%.

Performance comparison of CAIE against state-of-the-art pruning methods.

Beyond accuracy, CAIE demonstrated exceptional precision in meeting target resource budgets. While "constraint-blind" methods often overshoot or undershoot targets, CAIE's vector-based approach ensures all constraints are met simultaneously and accurately, even when constraints are lopsided (e.g., aggressive parameter reduction but mild FLOP reduction).

Conclusion

CAIE introduces a principled way to incorporate hardware costs directly into the filter pruning process. By treating pruning as an efficiency optimization problem, it enables the creation of compact models that are better suited for real-world deployment under complex, multi-dimensional resource constraints. Future work may involve extending the resource model to capture non-linear hardware latency more accurately.

Summary updated

BibTeX
@inproceedings{wu2020constraint,
  author = {Yu-Cheng Wu and Chih-Ting Liu and Bo-Ying Chen and Shao-Yi Chien},
  title = {{Constraint-Aware Importance Estimation for Global Filter Pruning under Multiple Resource Constraints}},
  booktitle = {Workshops @ CVPR},
  year = {2020},
  doi = {10.1109/cvprw50498.2020.00351},
}
Figure