YodaNN: An Architecture for Ultra-Low Power Binary-Weight CNN\n Acceleration

Renzo Andri, Lukas Cavigelli, Davide Rossi, Luca Benini

Abstract

Convolutional neural networks (CNNs) have revolutionized the world of computer vision over the last few years, pushing image classification beyond human accuracy. The computational effort of today's CNNs requires power-hungry parallel processors or GP-GPUs. Recent developments in CNN accelerators for system-on-chip integration have reduced energy consumption significantly. Unfortunately, even these highly optimized devices are above the power envelope imposed by mobile and deeply embedded applications and face hard limitations caused by CNN weight I/O and storage. This prevents the adoption of CNNs in future ultra-low power Internet of Things end-nodes for near-sensor analytics. Recent algorithmic and theoretical advancements enable competitive classification accuracy even when limiting CNNs to binary (+1/-1) weights during training. These new findings bring major optimization opportunities in the arithmetic core by removing the need for expensive multiplications, as well as reducing I/O bandwidth and storage. In this work, we present an accelerator optimized for binary-weight CNNs that achieves 1510 GOp/s at 1.2 V on a core area of only 1.33 MGE (Million Gate Equivalent) or 0.19 mm$^2$ and with a power dissipation of 895 {\mu}W in UMC 65 nm technology at 0.6 V. Our accelerator significantly outperforms the state-of-the-art in terms of energy and area efficiency achieving 61.2 TOp/s/[email protected] V and 1135 GOp/s/[email protected] V, respectively.

Introduction

Convolutional Neural Networks (CNNs) have redefined computer vision, but their computational intensity creates a "power wall" for Internet of Things (IoT) end-nodes. These devices often operate on extremely limited power budgets (sub-mW), whereas standard GPUs and mobile processors consume orders of magnitude more energy. The primary bottlenecks are the massive number of multiplications required for convolutions and the energy cost of moving weights from memory to the processor.

YodaNN addresses these challenges by exploiting Binary-Weight CNNs, where weights are restricted to +1 or -1. This simplification allows for radical hardware optimizations, replacing power-hungry multipliers with simple logic and enabling extreme voltage scaling to meet the requirements of near-sensor analytics.

Architecture and Methods

The YodaNN architecture is specifically optimized for the unique arithmetic of binary weights. Since weights are only +1 or -1, traditional multipliers are replaced with simple multiplexers and 2’s complement units. This reduces the area of the Sum-of-Product (SoP) units by over 5x compared to fixed-point implementations.

Block diagram of the YodaNN architecture, highlighting the FilterBank, ImageBank, and parallel Sum-of-Product (SoP) units.

Key technical innovations include:

  • Latch-Based Standard Cell Memory (SCM): Standard SRAM memory typically fails when supply voltage drops below 0.8V. YodaNN uses SCM for on-chip storage, which allows the chip to scale down to 0.6V. This "near-threshold" operation provides a 6x boost in energy efficiency compared to nominal 1.2V operation.
  • Flexible Kernel Support: YodaNN supports various kernel sizes (from 1x1 to 7x7) through a reconfigurable adder tree and zero-padding logic.
  • Data Reuse Strategy: It employs a sophisticated sliding window and "Image Bank" system to minimize off-chip memory access, ensuring that each pixel is loaded minimally while being reused across up to 32 parallel output channels.

Key Insights and Results

The shift to binary weights allows for a significant increase in compute density. As shown in the area analysis, the binary-weight units are substantially smaller than their fixed-point counterparts (Q2.9), allowing more parallel processing elements within the same silicon footprint.

Area comparison of YodaNN components across different configurations, demonstrating the significant area reduction of binary-weight units compared to fixed-point (Q2.9).

Evaluated on benchmarks like CIFAR-10, ImageNet (using AlexNet and ResNet), and the Stanford Backgrounds Dataset, YodaNN sets a new bar for efficiency in the 65nm process node:

  • Peak Performance: 1.5 TOp/s (Tera-Operations per second) at 1.2V.
  • Ultra-Low Power: Dissipates only 895 µW at 0.6V while maintaining functional throughput.
  • Efficiency: Achieves 61.2 TOp/s/W at 0.6V, a 32x improvement in peak energy efficiency over previous state-of-the-art accelerators like NINEX.
  • Real-World Throughput: It can process a ResNet-34 model at 16.8 FPS (at 1.2V) or smaller state-of-the-art CNNs at 11 FPS (at 0.6V) within a sub-mW budget.

Discussion and Conclusion

While YodaNN achieves remarkable efficiency, it is specialized for binary weights. While accuracy loss is often minimal for many tasks, some highly complex applications might still require higher precision. Additionally, while weights are binary, the activations are still 12-bit fixed-point. Future work could explore fully binary networks (like XNOR-Net) to yield even further gains. Furthermore, while SCM enables low-voltage operation, it is more area-intensive than SRAM; future research could investigate hybrid memory structures to better balance area and voltage scalability.

Summary updated

Abstract

Convolutional neural networks (CNNs) have revolutionized the world of\ncomputer vision over the last few years, pushing image classification beyond\nhuman accuracy. The computational effort of today's CNNs requires power-hungry\nparallel processors or GP-GPUs. Recent developments in CNN accelerators for\nsystem-on-chip integration have reduced energy consumption significantly.\nUnfortunately, even these highly optimized devices are above the power envelope\nimposed by mobile and deeply embedded applications and face hard limitations\ncaused by CNN weight I/O and storage. This prevents the adoption of CNNs in\nfuture ultra-low power Internet of Things end-nodes for near-sensor analytics.\nRecent algorithmic and theoretical advancements enable competitive\nclassification accuracy even when limiting CNNs to binary (+1/-1) weights\nduring training. These new findings bring major optimization opportunities in\nthe arithmetic core by removing the need for expensive multiplications, as well\nas reducing I/O bandwidth and storage. In this work, we present an accelerator\noptimized for binary-weight CNNs that achieves 1510 GOp/s at 1.2 V on a core\narea of only 1.33 MGE (Million Gate Equivalent) or 0.19 mm$^2$ and with a power\ndissipation of 895 {\\mu}W in UMC 65 nm technology at 0.6 V. Our accelerator\nsignificantly outperforms the state-of-the-art in terms of energy and area\nefficiency achieving 61.2 TOp/s/[email protected] V and 1135 GOp/s/[email protected] V, respectively.\n
BibTeX
@inproceedings{andri2016yodann,
  author = {Renzo Andri and Lukas Cavigelli and Davide Rossi and Luca Benini},
  title = {{YodaNN: An Architecture for Ultra-Low Power Binary-Weight CNN\textbackslash\{\}n Acceleration}},
  booktitle = {ISVLSI},
  year = {2016},
  doi = {10.1109/isvlsi.2016.111},
}
Figure