Abstract
Convolutional neural networks (CNNs) have revolutionized the world of computer vision over the last few years, pushing image classification beyond human accuracy. The computational effort of today's CNNs requires power-hungry parallel processors or GP-GPUs. Recent developments in CNN accelerators for system-on-chip integration have reduced energy consumption significantly. Unfortunately, even these highly optimized devices are above the power envelope imposed by mobile and deeply embedded applications and face hard limitations caused by CNN weight I/O and storage. This prevents the adoption of CNNs in future ultra-low power Internet of Things end-nodes for near-sensor analytics. Recent algorithmic and theoretical advancements enable competitive classification accuracy even when limiting CNNs to binary (+1/-1) weights during training. These new findings bring major optimization opportunities in the arithmetic core by removing the need for expensive multiplications, as well as reducing I/O bandwidth and storage. In this work, we present an accelerator optimized for binary-weight CNNs that achieves 1510 GOp/s at 1.2 V on a core area of only 1.33 MGE (Million Gate Equivalent) or 0.19 mm$^2$ and with a power dissipation of 895 {\mu}W in UMC 65 nm technology at 0.6 V. Our accelerator significantly outperforms the state-of-the-art in terms of energy and area efficiency achieving 61.2 TOp/s/[email protected] V and 1135 GOp/s/[email protected] V, respectively.
Introduction
We are currently facing a "CNN power wall." While deep learning has revolutionized computer vision, running these models usually requires power-hungry GPUs or high-end CPUs. For mobile and Internet of Things (IoT) devices—such as smart sensors running on tiny batteries—the computational cost and memory bandwidth required to move weights are prohibitive. Sourcing this work to the cloud is often impractical due to the massive radio power required for high-definition video transmission. To enable "near-sensor analytics," hardware must process neural networks locally within a milliwatt power budget. YodaNN addresses this by optimizing for binary-weight CNNs, drastically reducing the energy and area overhead of standard deep learning architectures.
Methods and Architecture
YodaNN introduces two major innovations to achieve its efficiency goals: BinaryConnect logic and Latch-Based Standard Cell Memory (SCM).
BinaryConnect Logic
Instead of using standard 12-bit or 16-bit weights, YodaNN utilizes binary weights (+1 and -1). This allows the hardware to replace expensive digital multipliers with simple multiplexers and sign-inversion logic, drastically reducing the silicon area and energy required for every operation. The architecture features 32 parallel Sum-of-Product (SoP) units and a flexible "Image Bank" that uses a sliding window approach to minimize redundant data movement.
Voltage Scaling and SCM
Traditional SRAM memory becomes unstable when operating below 0.8V. To push efficiency further, the team employed Latch-Based SCM. While SCMs require more physical space than SRAM, they remain stable at "near-threshold" voltages. This allowed the chip to operate at just 0.6V, where it is significantly more energy-efficient. The jump from 1.2V to 0.6V provided a 6x improvement in energy efficiency.
Key Results
The researchers evaluated YodaNN across industry-standard benchmarks including CIFAR-10, SVHN, and large-scale networks like ResNet-34 on ImageNet. The results demonstrate that binary weights result in minimal accuracy loss while providing massive hardware benefits.
- Peak Performance: 1.5 TOp/s (Tera-Operations per second) at 1.2V.
- Extreme Efficiency: 61.2 TOp/s/W at 0.6V, a 32x improvement over previous state-of-the-art accelerators.
- Physical Footprint: The core occupies just 0.19 mm² in a 65nm process.
- Real-world Throughput: The system can process ResNet-34 at 16.8 frames per second (FPS) at 1.2V, or run simpler networks at ~11 FPS while consuming less than a single milliwatt (895 µW).
Conclusion
YodaNN demonstrates that specialized hardware for binary-weight CNNs can overcome the power constraints of deeply embedded IoT devices. By eliminating multipliers and leveraging near-threshold voltage scaling, YodaNN achieves unprecedented energy efficiency. As the arithmetic core becomes increasingly efficient, future research must focus on I/O power—the energy cost of moving data in and out of the chip—which remains the next dominant bottleneck for near-sensor analytics.
Summary updated