Channel Pruning via Lookahead Search Guided Reinforcement Learning

Zi Wang, Chengcheng LiWACV 2022

TL;DR

This paper introduces a novel channel pruning method that combines reinforcement learning (RL) with Monte Carlo Tree Search (MCTS) to efficiently find optimal network architectures. By using MCTS for lookahead search, the approach significantly improves sample efficiency and narrows down the search space, leading to more compact and performant neural networks.

What Problem Does This Paper Address?

Deep Convolutional Neural Networks (CNNs) have achieved remarkable success in computer vision tasks, but their computational and storage demands make them unsuitable for resource-constrained devices. Channel pruning is a popular compression technique that aims to remove entire filters (channels) to reduce network size with minimal performance degradation. However, existing methods face two main challenges: 1) they rely on approximations to estimate filter importance due to the vast search space of filter combinations, and 2) they often use greedy selection strategies without considering the long-term impact of pruning decisions. This paper addresses these limitations by proposing a more systematic and efficient approach to channel pruning.

Methods

The core of this work is a novel channel pruning approach that integrates Reinforcement Learning (RL) with Monte Carlo Tree Search (MCTS).

  • Reinforcement Learning (RL) for Filter Selection: A neural network is trained using RL to act as a "pruning agent." This agent takes filter-related features extracted from the CNN as input and outputs two key components:
  • Policy ($\pi_t$): Probabilities of pruning each filter.
  • Value ($v_t$): An estimate of the performance of the pruned network. The RL agent learns to prune filters sequentially to maximize the performance of the resulting slimmed network.

  • Monte Carlo Tree Search (MCTS) for Lookahead: To overcome the limitations of greedy selection and improve sample efficiency, MCTS is employed. For each pruning decision, MCTS performs a lookahead search. It builds a search tree where nodes represent CNN configurations and edges represent pruning actions. By simulating potential pruning sequences, MCTS helps the RL agent make more informed decisions, effectively exploring a larger portion of the search space than a purely greedy approach.

  • Feature Extraction: To represent the state for the RL agent, various filter-related features are extracted, including:
  • Saliency responses (mean ReLU activation, average percentage of zero activations, estimated loss change).
  • Filter weights (l2-norm).
  • Structural information (number of filters in the layer, layer index).
  • Model Training: The RL policy network is trained using an advantage actor-critic (A2C) approach. Rewards are given based on whether the pruned network's performance improves compared to a baseline.
  • Search Space Reduction Strategies: To further enhance efficiency, two strategies are introduced:
  • Masking: Important filters identified by other criteria (e.g., Taylor expansion) can be masked out, preventing MCTS from considering them for pruning.
  • Segmentation: The overall pruning task is broken down into smaller segments, allowing for iterative pruning and fine-tuning, which reduces the depth of the MCTS search.

Datasets

The proposed approach was evaluated on the following datasets:

  • MNIST: For LeNet5.
  • CIFAR-10: For VGG-16 and ResNet-56.
  • ILSVRC-2012 (ImageNet): For ResNet-50.

The paper does not introduce new benchmarks.

Key Insights and Ideas

  • MCTS as a Game-Changer for Pruning: The integration of MCTS with RL is a key innovation. It transforms channel pruning from a potentially myopic greedy process into a more strategic search, significantly improving the quality of pruned networks and the efficiency of the search.
  • Bridging the Gap Between Oracle and Greedy Pruning: The paper effectively bridges the gap between computationally infeasible "oracle pruning" (evaluating all combinations) and often suboptimal greedy pruning by using MCTS to approximate a more thorough search.
  • Filter Importance is Dynamic: The work implicitly highlights that filter importance can change significantly during the pruning process, reinforcing the need for a lookahead mechanism like MCTS rather than relying solely on static importance scores.
  • Feature Engineering Matters: The selection of diverse filter-related features (saliency, weights, structural) is crucial for the RL agent to learn effective pruning policies.

Key Results

The proposed RL-MCTS approach demonstrated strong performance across various datasets and network architectures:

  • MNIST: Achieved a test accuracy of 99.28% while pruning 95.6% of FLOPs, outperforming previous methods like NISP [93] and GAL [51] in terms of both accuracy and compression ratio.
  • CIFAR-10 (VGG-16, ResNet-56): Reduced nearly 80% of FLOPs on VGG-16 with no performance loss, outperforming existing methods. For ResNet-56, it removed 55% FLOPs with a higher accuracy than baseline and other automated pruning approaches.
  • ILSVRC-2012 (ResNet-50): By pruning 46% FLOPs, the network's top-1 and top-5 accuracies dropped by only 0.54% and 0.27% respectively, showing competitive results on a large-scale dataset.
  • Ablation Studies:
    • Increasing training iterations improved both training and test accuracy.
  • More MCTS simulations led to better performance and pruning of filters in deeper layers.
  • Using training accuracy as an evaluation criterion performed slightly better than using loss.

Limitations and Future Work

  • Limitations:
    • The paper acknowledges that even with MCTS, the search space can still be large, and introduces strategies like masking and segmentation to mitigate this.
  • The performance of random search was unsatisfactory, highlighting the importance of the learned policy and MCTS.
  • The paper mentions that the approach is not sensitive to commonly used evaluation criteria, but a deeper dive into this might be beneficial.
  • Future Work:
    • The authors plan to focus on improving the masking and evaluation strategies.
  • Investigating the potential usage of this approach in a "zero-shot pruning" manner is a promising future direction.

Summary updated

BibTeX
@inproceedings{wang2022channel,
  author = {Zi Wang and Chengcheng Li},
  title = {{Channel Pruning via Lookahead Search Guided Reinforcement Learning}},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
  year = {2022},
}
Figure