Abstract
Deep learning models are increasingly deployed on resource-constrained devices, necessitating efficient model compression techniques. This paper presents scFusion, a novel approach that utilizes hand-crafted, spatially complementary (SC) sparse kernels to create compact Convolutional Neural Networks (CNNs). By fusing these structured sparse patterns in parallel, the proposed scFusion module achieves up to a 12.4× reduction in model size and a 3× speedup for object detection tasks with minimal impact on accuracy.
Introduction
The proliferation of deep learning in real-time IoT systems and mobile devices has driven the demand for smaller, faster CNNs. While traditional model compression techniques like pruning can significantly reduce parameter counts, they often result in irregular sparsity patterns. These "random" gaps are difficult for standard hardware like GPUs to exploit efficiently, as they are optimized for dense computations.
Problem Statement: The Challenge of Irregularity
Most existing compression methods face a practical hurdle: hardware-software mismatch. Irregularly pruned weights require specialized engineering to achieve actual wall-clock speedups. To address this, the authors propose moving away from learned random sparsity toward structured, hand-crafted sparsity that is inherently hardware-friendly.
Methodology: The scFusion Module
The core innovation is the scFusion module, which leverages Spatial Complementary (SC) kernels to maintain full receptive field coverage while reducing computation.
1. Checkerboard Kernels
Instead of standard dense kernels, scFusion employs pairs of sparse kernels, $W_{even}$ and $W_{odd}$, arranged in a checkerboard pattern. $W_{even}$ contains weights only on even-indexed positions, while $W_{odd}$ occupies the odd positions.
2. Parallel Kernel Fusion
Unlike sequential stacking of sparse kernels, which can lead to "blind spots" in the receptive field, scFusion processes these kernels in parallel. Their outputs are added together, ensuring that every pixel in the receptive field contributes to the final feature map.
3. Inverse and Channel-wise Fusion
To enrich feature representation without significantly increasing parameters, an "inverse" operation is applied to generate symmetric feature maps. Finally, a $1 \times 1$ pointwise convolution linearly combines these sparse and joint features into a robust final representation.
Experimental Results
The scFusion approach was validated on benchmarks including CIFAR-10, ImageNet, PASCAL VOC, and MS COCO across various architectures like AlexNet, VGG-16, and ResNet.
- Model Efficiency: On ImageNet, a VGG-16 model enhanced with scFusion-8 and Deep Compression was 13.45× smaller than the baseline.
- Object Detection Performance: In Faster R-CNN, the scFusion variant achieved a 3× speedup and 12.4× size reduction compared to a VGG-16 baseline, maintaining a competitive 73.4 mAP.
- Hardware Acceleration: The deterministic nature of SC kernels allowed for a ~2× real-world speedup on an Nvidia K80 GPU, demonstrating its practical utility.
Conclusion
scFusion offers a compelling balance between efficiency and performance by utilizing structured sparsity. By ensuring full receptive field coverage through parallel fusion and leveraging hardware-friendly patterns, it provides a scalable solution for deploying high-performance CNNs on edge devices. Future work may explore combining scFusion with advanced quantization and distillation techniques for even greater compression.
Summary updated