Abstract
Hyperdrive is a novel hardware accelerator designed to bring high-resolution computer vision to power-constrained IoT devices. By implementing a feature-map stationary dataflow—streaming 1-bit binary weights while keeping feature maps on-chip—and a systolically scalable multi-chip architecture, Hyperdrive achieves a system-level energy efficiency of 4.3 TOp/s/W. This represents a 3.1x improvement over state-of-the-art accelerators, enabling the deployment of complex models like ResNet and YOLOv3 in milliwatt-scale environments.
Introduction
Deep Neural Networks (DNNs) are essential for modern computer vision but pose significant challenges for edge deployment due to high power consumption. Binary-Weight Networks (BWNs) mitigate memory constraints by compressing weights to 1 bit. However, traditional accelerators often hit an "energy wall" caused by excessive I/O energy spent moving large feature maps (FMs) between on-chip and off-chip memory. Most existing designs either stream FMs, consuming massive I/O energy, or store everything on-chip, which limits the device to low-resolution models. Hyperdrive addresses this by flipping the dataflow priority and introducing a scalable multi-chip design.
Architecture and Methodology
The core innovation of Hyperdrive is its Feature-Map Stationary approach. In a typical BWN, activations (feature maps) are significantly larger than the 1-bit weights. Hyperdrive keeps these large feature maps in on-chip memory and "streams" the compressed binary weights from external memory. This "I/O flip" reduces system-level energy consumption by approximately 25% compared to designs that move activations.
The system is systolically scalable, allowing multiple chips to be arranged in a 2D mesh. Each chip processes a spatial "tile" of the input image and exchanges border pixels through a dedicated interface. This allows the system to scale to arbitrarily large images by adding more low-cost chips, making it suitable for applications ranging from simple doorbell cameras to high-end autonomous drones.
Each chip contains a 7x7 grid of Tile Processing Units (TPUs). These units utilize FP16 (16-bit floating point) for activations. While more resource-intensive than fixed-point math, FP16 provides the robustness needed for deep networks, and the FM-stationary approach makes the extra bits affordable in terms of energy. A latency-hiding dataflow mechanism using ping-pong buffering ensures that weights are loaded while computation proceeds, maintaining high hardware utilization.
Experimental Evaluation
Hyperdrive was evaluated using industry-standard benchmarks, including ImageNet (for ResNet-18/34) and COCO (for YOLOv3 object detection). Implemented in 22nm FDX technology, a single chip reaches a throughput of 88 GOp/s at 0.5V.
Key results include:
- Efficiency: Achieved 4.3 TOp/s/W at the system level, a 3.1x improvement over previous state-of-the-art BWN accelerators.
- Utilization: Maintained over 97% utilization for ResNet-34, minimizing idle hardware cycles.
- Scalability: Simulations demonstrated that a 20x10 mesh of chips could execute a ResNet-152 model on high-definition (2k x 1k) images while maintaining high efficiency.
Discussion and Conclusion
Hyperdrive proves that optimizing data movement is as critical as optimizing computation for edge AI. By treating weights as the transient stream and activations as stationary, it overcomes the I/O energy-induced efficiency wall. While the current design is optimized for 1x1 and 3x3 convolutions—requiring external handling for larger kernels like the initial 7x7 layer in ResNet—it offers a highly scalable and efficient solution for bringing advanced computer vision to the IoT frontier. Future work may address kernel size constraints and explore physical packaging impacts on I/O energy.
Summary updated