TL;DR
This paper introduces AutoSculpt, a novel framework that uses graph learning and deep reinforcement learning to automatically prune deep neural networks (DNNs). By representing DNNs as graphs and identifying "regular patterns" for pruning, AutoSculpt achieves significant model compression with minimal accuracy loss, making DNNs more deployable on resource-constrained edge devices.
What Problem Does This Paper Address?
Deploying powerful deep neural networks (DNNs) on edge devices like smartphones and robots is a major challenge due to their high computational demands and limited resources. Model compression techniques, particularly pruning, aim to reduce model size and computational cost. However, existing auto-pruning methods struggle with the diversity of DNN architectures and operators, and often face a trade-off between pruning granularity and model accuracy. Specifically, unstructured pruning (removing individual weights) is hard to accelerate on hardware, while structured pruning (removing entire filters or channels) can be too coarse and lead to significant accuracy drops. Pattern-based pruning offers a middle ground by removing specific "regular structures" within weight tensors, but this area is underexplored and lacks generalizability across different DNN types, especially newer architectures like Transformers.
Methods
AutoSculpt tackles these challenges with a three-step approach:
- Graph Construction: DNNs are converted into graphs. Nodes represent operators (like convolutional layers) and their dependencies, while edges capture relationships. For CNNs, this involves mapping weights, input/output feature maps, and even residual connections to graph components. For Transformers, it models attention mechanisms and feed-forward networks.
- Pattern-based Pruning: The framework integrates "pruning patterns" into the graph. These are computationally friendly structures that can be efficiently recognized by inference engines. The patterns are embedded into edge features of the graph.
- Reinforcement Learning for Strategy Search: A Graph Neural Network (GNN), specifically GATv2, encodes the graph to capture topological and parameter dependencies. This embedding is then fed to a Deep Reinforcement Learning (DRL) agent. The agent learns an optimal pruning policy by iteratively selecting patterns to prune, aiming to balance compression (FLOPs reduction) and accuracy. A reward function is designed to encourage aggressive compression while penalizing accuracy loss.
Datasets
The paper evaluates AutoSculpt on standard image classification datasets:
- CIFAR-10/100
- ImageNet-1K (ILSVRC-2012)
Key Insights and Ideas
- Graph Representation is Key: Representing DNNs as graphs, especially incorporating topological information and dependencies, is crucial for effective automated pruning. This allows the system to understand the network's structure beyond just individual weights.
- Pattern-based Pruning for Hardware Efficiency: Identifying and pruning "regular patterns" is a promising direction for achieving both high compression and hardware acceleration, bridging the gap between unstructured and structured pruning.
- DRL for Optimal Strategy Discovery: Deep Reinforcement Learning is a powerful tool for automatically discovering complex pruning strategies that balance multiple objectives (compression, accuracy) across diverse network architectures.
- Generalizability Across Architectures: AutoSculpt's graph-based approach allows it to handle various DNN architectures, including CNNs (ResNet, MobileNet, VGG) and Transformers, which is a significant improvement over prior methods.
Key Results
AutoSculpt demonstrates strong performance across various architectures:
- High Pruning Rates: Achieves pruning rates of up to 90% (e.g., on VGG-19).
- Significant FLOPs Reduction: Achieves nearly 18% improvement in FLOPs reduction compared to the second-best auto-pruning method (GNN-RL on ResNet-50).
- Minimal Accuracy Loss: Maintains accuracy levels close to the original models after fine-tuning, with accuracy loss generally under 3% for many models. In some cases (ResNet-110, MobileNet-v1), accuracy even slightly improved.
- Latency Reduction: Demonstrates significant reductions in inference latency, for example, reducing latency for VGG-19 from 5.3 ms to 4.1 ms (nearly 29% speed improvement) while maintaining comparable accuracy.
- Effectiveness of Graph Encoders: Ablation studies show that using GCN or GAT encoders significantly improves pruning performance compared to methods without graph encoders, with GAT yielding slightly better results.
Limitations and Future Work
- Pattern Generation: The current framework manually defines up to 10 pruning patterns. Future work aims to improve pattern generation methods and expand the library.
- Wider DNN Support: While AutoSculpt shows promise for CNNs and Transformers, future work could focus on expanding its applicability to a wider range of DNN types.
- Hyperparameter Sensitivity: The paper notes that the number of patterns and node embedding size can impact performance, suggesting further exploration into optimal hyperparameter tuning.
Summary updated