Introduction
Deep neural network (DNN) compression is essential for deploying state-of-the-art models on resource-constrained edge devices. Pruning is a primary technique for this, yet it traditionally faces a stark tradeoff. Fine-grained (unstructured) pruning maintains high accuracy but results in irregular memory access patterns that hinder hardware acceleration. Conversely, structured pruning (removing entire filters) is hardware-friendly but often leads to significant accuracy degradation due to its rigid constraints. This paper proposes Cross-Channel Intragroup (CCI) Sparsity, a novel pruning structure designed to bridge this gap by offering dense-like hardware efficiency with the accuracy of fine-grained methods.
Cross-Channel Intragroup (CCI) Sparsity
The core innovation is the CCI-Sparsity pattern. Unlike unstructured pruning, CCI-Sparsity partitions weights into small, fixed-size groups across output channels. For a given group size $G$, exactly $s$ weights remain non-zero. For instance, with $G=4$ and $s=1$, one weight is selected from the same spatial position across four consecutive output channels.
This structure is mathematically optimized for memory I/O. By grouping across output channels rather than input channels, the hardware can maintain output registers in a stationary state while reading input and weight matrices in continuous blocks. This eliminates the overhead of irregular dataflow and partial sum management typical of other sparse formats.
Optimized Training: Ramping Targeted Dropout
Training networks under the strict constraints of CCI-Sparsity is challenging. Traditional iterative pruning can be suboptimal as it permanently removes weights that might have become important later. The authors employ Targeted Dropout (TD), where low-magnitude weights are stochastically dropped during training.
A key contribution of this work is an improved TD algorithm. The authors identified that standard TD (with a fixed 50% dropout rate) allows "pruned" weights to hover near zero without reaching it, which interferes with Batch Normalization (BN) statistics and degrades accuracy. To solve this "silent killer," they introduce a ramping dropout rate that increases from 50% to 100% over the training duration. This forces pruned weights to exactly zero, ensuring BN layers reflect the true sparse architecture.
Experimental Results
The proposed method was evaluated on CIFAR-10 and ImageNet across various architectures including ResNet, VGG, and MobileNetV2.
- ImageNet Performance: On MobileNetV2, CCI-Sparsity ($G=8, s=2$, equating to 75% sparsity) achieved 69.8% Top-1 accuracy with ramping dropout, a 5% improvement over fixed-rate dropout.
- Comparison to Structured Pruning: On ResNet-18, the method achieved 70.31% accuracy while pruning roughly 70% of FLOPs. This significantly outperforms traditional structured pruning methods, which typically see accuracy drops of over 3% for much lower pruning ratios.
- Pareto Frontier: CCI-Sparsity models consistently outperformed specialized lightweight architectures like ShuffleNetV2 and CondenseNet in Accuracy vs. FLOPs comparisons.
Conclusion
CCI-Sparsity provides a compelling middle ground in the pruning landscape. By leveraging output-channel grouping for hardware efficiency and a ramping Targeted Dropout strategy for training stability, it enables high-performance sparse models that are truly practical for edge deployment. Future work includes combining CCI-Sparsity with coarse-grained pruning and exploring advanced regularization techniques like Exclusive Lasso to further enhance convergence.
Summary updated