Decoupling efficiency-performance optimization for modern neural networks

Qing Jin2023 · DOI

Abstract

Deep learning has achieved significant success in a variety of modern applications, but this remarkable achievement is often accompanied by inefficiency in terms of memory storage, computing resources, and inference speed. Such inefficiency prevents wide applications of modern deep models in practical problems, especially on resource-constrained computing platforms. Particularly, developing highly efficient neural networks while maintaining high prediction accuracy with much-reduced computing requirements is challenging and urgent. Meanwhile, theoretical analysis and an intuitive understanding of the underlying mechanism of performance bottleneck of efficient models would provide practical guidance and technical solutions to alleviate trade-offs between accuracy and efficiency. In addition, for more special cases, such as some specific hardware with non-conventional computing architectures, the problem can be more complicated. Such hardware has the potential of much higher energy efficiency and much faster inference speed, but the harmful effects induced by hardware non-idealities, such as limited computing precision and storage size, as well as noise and nonlinearity inherent in the analog interface circuits. Software solutions for these hardware effects can be more economical and can reduce the iteration period. In this thesis, we will explore the potential for simultaneously achieving high efficiency and high prediction accuracy in neural networks, with several different techniques and theoretical analyses, for both conventional and innovative computing platforms. In the first part, we explore the implementation of highly efficient generative adversarial networks (GANs) to guarantee their capability of generating high-quality images within a predefined computational budget. The key challenges lie in identifying the optimal architecture for the generative models, while preserving the quality of the generated images from the compressed models, despite their reduced computational cost and storage capability. To achieve this goal, we introduce inception blocks into the generative models to enlarge the searching space and propose a novel neural architecture search (NAS) algorithm with much faster searching speed. Also, we design a new knowledge distillation technique, where extra layers for distillation are avoided. With our proposed techniques, we achieve better quality for generated images than the original huge models. Our work is the first in the world to achieve better image generation quality than the original huge model, demonstrating that for generative models, decoupling the optimization of high performance and high efficiency can be technically achieved. In the second part, we investigate quantization techniques for deep neural networks and study the necessity of high-precision multiplications in quantized models. Specifically, we study if we could achieve the same accuracy for classification tasks without using either floating-point multiplication or the 32-bit fixed-point multiplication for the scaling operation. We find that by properly training, we are able to achieve the same performance with 8-bit fixed-point multiplication alone. Specifically, we conduct some statistical analysis of weights and activations, and based on the semi-empirical analysis, we propose an algorithm to determine the fixed-point formats automatically during training. Also, we study how to fuse batch normalization layers with the associated convolution layers and derive the effective weights after layer fusion. For models with residual connections, we find that for those sibling layers with the same preceding layer, assigning private fractional lengths to them is crucial to achieving good performance. We demonstrate that high-precision multiplication is not indispensable for quantized neural networks to achieve good accuracy. In the third part, we study the problem of training neural networks for innovative computing platforms, especially in-memory computing systems. To this end, we study the extra quantization step caused by limited precision in the computing platform and analyze how to make the operation differentiable to facilitate the optimization process. We also study the impact of PIM quantization on the output scale, and by analyzing the training dynamics, we propose rescaling techniques to achieve convergent training. For non-ideal effects of noise and nonlinearity introduced by the analog converting interface, we adopt batch normalization calibration to reduce their impact. Our proposed framework is generic and applicable for different PIM quantization schemes, including native, differential, and bit-serial, and is verified with a real-chip prototype.--Author's abstract
BibTeX
@misc{jin2023decoupling,
  author = {Qing Jin},
  title = {{Decoupling efficiency-performance optimization for modern neural networks}},
  year = {2023},
  doi = {10.17760/d20592335},
}

Loading related content...

Figure