TL;DR
This paper introduces ABCP, a novel deep reinforcement learning (DRL) framework that automates the pruning of both residual blocks and individual channels in deep neural networks. By jointly searching for these pruning actions in a combined discrete and continuous space, ABCP achieves significantly higher compression ratios with minimal accuracy loss compared to traditional methods, making deep learning models more deployable on resource-constrained devices.
What Problem Does This Paper Address?
Deep learning models have become incredibly powerful but also computationally expensive, making them unsuitable for deployment on devices with limited resources like mobile phones or embedded systems. Model pruning aims to reduce this computational burden by removing redundant parts of the network. However, existing pruning methods often struggle to achieve high compression rates without sacrificing accuracy, are time-consuming due to manual hyperparameter tuning and iterative processes, or only focus on pruning channels, neglecting structural redundancies like entire blocks. This paper addresses the need for an automated, efficient, and more comprehensive pruning approach that can simultaneously optimize both structural (block-wise) and granular (channel-wise) redundancies.
Methods
The core of ABCP is a deep reinforcement learning (DRL) agent that learns to make pruning decisions. The agent uses a joint sample algorithm, implemented with a Stacked Long Short-Term Memory (LSTM) network, to generate a pruning action. This action is a list that specifies:
- Block-wise pruning: Whether to prune entire residual blocks (a discrete choice: prune or not prune).
- Channel-wise pruning: The ratio at which to prune channels within convolutional layers (a continuous choice, allowing for fine-grained control).
The DRL agent is trained using a policy gradient method. The reward function for the agent balances the accuracy (measured by testing loss) and the computational cost (measured by FLOPs) of the pruned model. The process involves:
- Sampling: The LSTM generates a pruning action.
- Pruning & Fine-tuning: The model is pruned according to the action, and then specific layers are fine-tuned to recover accuracy.
- Updating: The reward is calculated, and the DRL agent's parameters are updated.
- Re-training: After training the DRL agent, the best pruning action is selected, and the final pruned model is re-trained from scratch.
A key innovation is the joint search space, combining discrete choices for block pruning and continuous choices for channel pruning, which the authors argue is more effective than separate or purely discrete/continuous approaches.
Datasets
The paper evaluates ABCP on three datasets:
- UCSD dataset: A small dataset captured from freeway surveillance videos, featuring sparse, medium, and dense traffic. It includes three classes: truck, car, and bus.
- Mobile robot detection dataset: Collected from robot-mounted cameras, designed for fast and lightweight detection. It features five classes: red robot, red armor, blue robot, blue armor, and dead robot.
- Sim2real detection dataset: Consists of two parts: a simulation dataset (generated using Gazebo) and a real-world dataset. The model is trained on the simulation data and tested on the real-world data to evaluate robustness and transferability. This dataset has one class: robot.
The authors also compare their method against YOLOv3, YOLOv4, YOLO-tiny, and a rule-based pruning method (RBCP) trained on these datasets.
Key Insights and Ideas
- Joint Block and Channel Pruning is Crucial: Pruning both entire residual blocks and individual channels simultaneously leads to significantly better compression and accuracy than pruning only one aspect. Block pruning offers coarse-grained reduction, while channel pruning provides fine-grained optimization.
- Continuous Channel Pruning is Superior: Using a continuous search space for channel pruning ratios is more effective than discrete options, allowing for more sensitive and accurate pruning decisions that better preserve accuracy.
- DRL for Automated Pruning: Deep reinforcement learning provides an effective mechanism to automate the complex search for optimal pruning strategies, reducing human effort and potentially discovering better pruning policies than manual or rule-based methods.
- YOLOv3's Redundancy: The experiments suggest that YOLOv3, and even YOLOv4, contain substantial redundancy, especially on simpler detection tasks, which ABCP effectively exploits for high compression.
Key Results
- On the mobile robot detection dataset: ABCP pruned YOLOv3 achieved a remarkable 99.5% reduction in FLOPs and parameters, resulting in a 37.3x speedup with only a 2.8% mAP loss. This performance was comparable to YOLOv4 but with vastly fewer resources.
- On the UCSD dataset: ABCP improved mAP by 8.2% over the baseline YOLOv3, with 93.2% FLOPs reduction and 92.4% parameter reduction, achieving a 6.87x speedup. It also outperformed YOLOv4.
- On the sim2real detection dataset (transfer task): ABCP achieved 2.4% better accuracy on the simulation dataset and 9.6% better accuracy on the real-world dataset compared to baseline YOLOv3, with a 97.6% FLOPs reduction and 14.6x speedup. This highlights ABCP's effectiveness in improving robustness.
- Comparison with RBCP: ABCP consistently achieved higher compression ratios with better or comparable accuracy compared to the rule-based pruning method (RBCP).
Limitations and Future Work
- Limited to Detection Models: The current work focuses on object detection models, specifically YOLOv3. The authors express a desire to extend ABCP to other deep learning tasks like image classification and 3D object detection.
- Re-training Strategy: While the paper mentions re-training the final pruned model from scratch, it notes that pruning and re-training all sampled candidate models might yield better results but is too time-consuming. Future work could explore more efficient ways to leverage multiple candidate pruned models.
- Hyperparameter Tuning: Although ABCP automates pruning, parameters like the reward trade-off hyperparameter ($\lambda$) still require tuning.
Summary updated