AddressNet: Shift-Based Primitives for Efficient Convolutional Neural Networks

Yihui He, Xianggen Liu, Huasong Zhong, Yuchun MaWACV 2019 · DOI

Introduction: The Memory-Bound Bottleneck

In the design of compact Convolutional Neural Networks (CNNs), a significant gap exists between theoretical efficiency (measured in FLOPs) and actual hardware performance. While architectures like MobileNet and ShuffleNet reduce computation through depthwise separable convolutions and channel shuffling, these operations often become "memory-bound" on GPUs. Operations that require zero FLOPs, such as channel shuffling or identity mappings, can consume up to 30% of total inference time due to heavy data movement in memory. This paper addresses this paradox by proposing methods to minimize physical data movement.

Proposed Shift-based Primitives

To bridge the gap between theoretical and practical speed, the authors introduce three "shift-based primitives" that replace memory-heavy operations with efficient pointer arithmetic. By manipulating memory addresses instead of copying data, these primitives achieve significant speedups without sacrificing accuracy.

Overview of the three proposed shift-based primitives: Channel Shift, Address Shift, and Shortcut Shift.

1. Channel Shift

Channel Shift serves as a high-speed alternative to the standard Channel Shuffle used in group convolutions. Traditional shuffling requires complex reordering of all channels, which is computationally expensive on GPUs due to fragmented memory access. In contrast, Channel Shift utilizes pointer shifting and a minimal circular memory copy at the boundaries to fuse information between groups.

Comparison between standard Channel Shuffle and the proposed Channel Shift, highlighting the use of pointer shifts and inplace operations.

As shown above, Channel Shift is significantly more efficient because it avoids massive memory reordering. On a GTX 1080 GPU, this operation is 12.7x faster than standard shuffling while maintaining identical accuracy.

2. Address Shift

Address Shift replaces traditional spatial convolutions. Instead of performing multiplication and addition (FLOPs), it simply offsets the memory pointer to "read" adjacent pixels in four fundamental directions: Up, Down, Left, and Right. This allows the network to capture spatial context with zero parameters and zero FLOPs, effectively providing the benefits of a 3x3 convolution at a fraction of the cost.

3. Shortcut Shift

Shortcut Shift optimizes residual connections. By pre-allocating continuous memory blocks, the output of one layer is written directly into the memory space expected by the subsequent layer. This makes identity mappings and concatenations virtually "free" in terms of both computation and memory overhead.

AddressNet Architectures

These primitives are integrated into two main building blocks: the Address-based module and the Address-enhanced module. These modules form the basis of AddressNet and Enhanced AddressNet, architectures designed for maximum practical efficiency on modern hardware.

Key Results

The proposed methods were evaluated on ImageNet and CIFAR-100, demonstrating superior performance-to-latency ratios:

  • Speed and Accuracy: Enhanced AddressNet-A achieved 67.0% Top-1 accuracy on ImageNet with a latency of only 5.6ms, outperforming several ShuffleNet and SqueezeNet variants in the same speed bracket.
  • Parameter Efficiency: AddressNet-44 outperformed ShiftResNet on CIFAR-100 while using 6x fewer parameters and 3x fewer FLOPs.
  • Drop-in Improvements: Integrating Address Shift into MobileNet (creating "Address-MobileNet") reduced latency from 2.0ms to 1.7ms while boosting accuracy from 41.3% to 46.0% in specific configurations.

Conclusion

The shift-based primitives demonstrate that pointer arithmetic is a powerful tool for optimizing CNNs. By focusing on data movement rather than just FLOPs, AddressNet architectures achieve state-of-the-art results on actual hardware. Future work will focus on specialized library optimizations to further unlock the potential of group convolutions and pointer-based operations.

Summary updated

BibTeX
@inproceedings{he2019addressnet,
  author = {Yihui He and Xianggen Liu and Huasong Zhong and Yuchun Ma},
  title = {{AddressNet: Shift-Based Primitives for Efficient Convolutional Neural Networks}},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
  year = {2019},
  doi = {10.1109/wacv.2019.00134},
}

Loading related content...

Figure