Abstract
This paper introduces channel gating, a dynamic, fine-grained, and hardware-efficient pruning scheme to reduce the computation cost for convolutional neural networks (CNNs). Channel gating identifies regions in the features that contribute less to the classification result, and skips the computation on a subset of the input channels for these ineffective regions. Unlike static network pruning, channel gating optimizes CNN inference at run-time by exploiting input-specific characteristics, which allows substantially reducing the compute cost with almost no accuracy loss. We experimentally show that applying channel gating in state-of-the-art networks achieves 2.7-8.0$\times$ reduction in floating-point operations (FLOPs) and 2.0-4.4$\times$ reduction in off-chip memory accesses with a minimal accuracy loss on CIFAR-10. Combining our method with knowledge distillation reduces the compute cost of ResNet-18 by 2.6$\times$ without accuracy drop on ImageNet. We further demonstrate that channel gating can be realized in hardware efficiently. Our approach exhibits sparsity patterns that are well-suited to dense systolic arrays with minimal additional hardware. We have designed an accelerator for channel gating networks, which can be implemented using either FPGAs or ASICs. Running a quantized ResNet-18 model for ImageNet, our accelerator achieves an encouraging speedup of 2.4$\times$ on average, with a theoretical FLOP reduction of 2.8$\times$.
Introduction
Deep Convolutional Neural Networks (CNNs) have become the standard for computer vision tasks but remain notoriously resource-intensive, complicating deployment on mobile and embedded devices. Traditional static pruning methods remove weights or filters permanently. While effective, these methods are "input-agnostic"—they apply the same computational budget to a simple image of a clear sky as they do to a complex street scene.
Dynamic pruning addresses this by skipping computations at runtime based on the input. However, previous dynamic approaches were often "coarse-grained" (skipping entire layers or channels) or introduced significant overhead by requiring auxiliary neural networks to make pruning decisions. Channel Gating solves this by providing a fine-grained (activation-specific), low-overhead, and hardware-efficient dynamic pruning mechanism.
Methodology: The Channel Gating Block
The core innovation is the Channel Gating Block. Instead of performing a standard full convolution, the layer's computation is split into two distinct paths:
- The Base Path: This path computes a partial sum using a small subset (e.g., 1/8th or 1/4th) of the input channels ($X_p$).
- The Gate: A lightweight Heaviside step function evaluates the partial sum. If the partial sum indicates that the final activation will likely be "ineffective" (e.g., clipped to zero by a ReLU), the gate remains closed.
- The Conditional Path: The convolution for the remaining input channels ($X_r$) is computed only if the gate is open.
Technical Implementation
- Learnable Thresholds: Gating thresholds are trained alongside network weights to optimize the trade-off between accuracy and sparsity.
- Differentiable Approximation: To handle the non-differentiable step function during training, the authors use a smooth sigmoid-like approximation for backpropagation.
- Channel Grouping: To prevent bias in the "base path" weights, the specific channels assigned to the base path are rotated across different groups.
- Hardware Co-design: The authors developed a specialized accelerator for systolic arrays (similar to Google’s TPU), demonstrating that this fine-grained sparsity translates into real-world hardware speedups.
Key Insights: Exploiting Spatial Redundancy
A critical discovery in this work is that partial sums are highly predictive of final sums, with Pearson correlation coefficients ranging from 0.56 to 0.86. This "sneak peek" allows the network to make accurate decisions about whether to skip the remaining computation.
Furthermore, by making decisions at the individual activation level rather than the channel level, the network can exploit spatial redundancy. For instance, the network can skip computations for background pixels while maintaining high precision for the main object of interest.
Experimental Results
The researchers evaluated Channel Gating on standard benchmarks including CIFAR-10 (ResNet-18, VGG, MobileNetV1) and ImageNet (ResNet-18, ResNet-34, MobileNetV1).
- Efficiency Gains: The method achieved a 2.7x to 8.0x reduction in FLOPs on CIFAR-10 and a 2.6x reduction on ImageNet for ResNet-18.
- Memory Savings: Off-chip memory accesses for weights were reduced by 2.0x to 4.4x.
- Hardware Performance: An ASIC accelerator prototype achieved a 2.4x average speedup on ImageNet, closely matching the theoretical FLOP reduction.
- Knowledge Distillation (KD): By using a larger "teacher" network to guide the gated "student" network, the authors achieved significant compute reduction on ImageNet with zero accuracy loss.
Conclusion
Channel Gating offers a powerful alternative to static pruning by leveraging input-specific characteristics at runtime. While standard GPUs and CPUs may require custom kernels to fully realize these speedups, the authors' hardware co-design proves the feasibility of the approach for specialized AI accelerators. Future work may extend these techniques to tasks like object detection, where spatial redundancy is even more pronounced.
Summary updated