Simulated Quantization, Real Power Savings

Mart van Baalen, Brian Kahne, Eric Mahurin, Andrey Kuzmin, Andrii Skliar, Markus Nagel, Tijmen BlankevoortWorkshops @ CVPR 2022 · OpenReview

Abstract

Reduced precision hardware-based matrix multiplication accelerators are commonly employed to reduce power consumption of neural network inference. Multiplier designs used in such accelerators possess an interesting property: When the same bit is 0 for two consecutive compute cycles, the multiplier consumes less power. In this paper we show that this effect can be used to reduce power consumption of neural networks by simulating low bit-width quantization on higher bit-width hardware. We show that simulating 4 bit quantization on 8 bit hardware can yield up to 17% relative reduction in power consumption on commonly used networks. Furthermore, we show that in this context, bit operations (BOPs) are a good proxy for power efficiency, and that learning mixed-precision configurations that target lower BOPs can achieve better trade-offs between accuracy and power efficiency.
BibTeX
@inproceedings{baalen2022simulated,
  author = {Mart van Baalen and Brian Kahne and Eric Mahurin and Andrey Kuzmin and Andrii Skliar and Markus Nagel and Tijmen Blankevoort},
  title = {{Simulated Quantization, Real Power Savings}},
  booktitle = {Workshops @ CVPR},
  year = {2022},
  url = {https://openreview.net/forum?id=8PjwiK7tBs},
}

Loading related content...

Figure