Abstract
Currently, Machine Learning (ML) is becoming ubiquitous in everyday life. Deep Learning (DL) is already present in many applications ranging from computer vision for medicine to autonomous driving of modern cars as well as other sectors in security, healthcare, and finance. However, to achieve impressive performance, these algorithms employ very deep networks, requiring a significant computational power, both during the training and inference time. A single inference of a DL model may require billions of multiply-and-accumulated operations, making the DL extremely compute- and energy-hungry. In a scenario where several sophisticated algorithms need to be executed with limited energy and low latency, the need for cost-effective hardware platforms capable of implementing energy-efficient DL execution arises. This paper first introduces the key properties of two brain-inspired models like Deep Neural Network (DNN), and Spiking Neural Network (SNN), and then analyzes techniques to produce efficient and high-performance designs. This work summarizes and compares the works for four leading platforms for the execution of algorithms such as CPU, GPU, FPGA and ASIC describing the main solutions of the state-of-the-art, giving much prominence to the last two solutions since they offer greater design flexibility and bear the potential of high energy-efficiency, especially for the inference process. In addition to hardware solutions, this paper discusses some of the important security issues that these DNN and SNN models may have during their execution, and offers a comprehensive section on benchmarking, explaining how to assess the quality of different networks and hardware systems designed for them.
Introduction: The Computational Cost of Intelligence
Deep Learning has revolutionized fields from medical imaging to autonomous driving, but its "intelligence" comes at a high cost. Modern Deep Neural Networks (DNNs) require billions of Multiply-and-Accumulate (MAC) operations for a single inference, making them incredibly compute- and energy-hungry. This survey addresses the "Von Neumann Bottleneck"—the energy-intensive process of moving data between memory and processors—and the mismatch between general-purpose hardware and the specific needs of neural networks. As we move toward "AI at the Edge" (IoT, wearables, and robotics), we need systems that provide high performance within a strict power envelope.
Survey Structure and Scope
The survey provides a multi-layered analysis of the optimization landscape, from fundamental neural models to specialized hardware and benchmarking frameworks.
Hardware Architectures: From Flexibility to Efficiency
A central theme in hardware optimization is the trade-off between design flexibility and operational efficiency. The survey distinguishes between two primary architectural paradigms:
- Temporal Architectures (CPUs/GPUs): These use SIMD (Single Instruction, Multiple Data) or SIMT (Single Instruction, Multiple Threads) to process data sequentially. While highly flexible and easy to program, they often struggle with the energy costs of frequent memory access.
- Spatial Architectures (FPGAs/ASICs): These utilize arrays of Processing Elements (PEs) to move data directly between units, minimizing expensive DRAM accesses. They offer superior energy efficiency (GFLOPS/W) and are increasingly preferred for edge deployment.
Algorithmic Compression and Software Optimizations
Beyond hardware, software-level optimizations are critical for reducing the computational load:
- Quantization: Reducing weight and activation precision (e.g., from 32-bit float to 8-bit fixed-point) can yield up to 20x energy savings with negligible accuracy loss.
- Pruning: Removing redundant weights or neurons creates sparse networks, allowing specialized hardware to "skip" zero-value computations.
- Dataflow Optimization: Techniques like Weight Stationary or Row Stationary dataflows maximize data reuse within local buffers, further mitigating the "memory wall."
The Promise of Spiking Neural Networks (SNNs)
SNNs represent a bio-inspired approach to AI, using event-driven "spikes" to transmit information. This sparsity allows for extreme energy efficiency on neuromorphic hardware. However, SNNs are currently more difficult to train than traditional DNNs because their activation functions are non-differentiable. Current research focuses on hybrid training methods and DNN-to-SNN conversion to bridge this gap.
Benchmarking and Security
Standardized evaluation is essential for comparing diverse hardware-software solutions. The survey reviews classic datasets (MNIST, ImageNet) and specialized benchmarks like CORe50 for continual learning. It also highlights a critical trade-off: while optimizations like quantization improve efficiency, they can sometimes increase a model's vulnerability to adversarial attacks, necessitating a security-conscious co-design approach.
Conclusion and Future Directions
The road ahead for AI acceleration lies in the tight integration of hardware and software. Hardware-aware Neural Architecture Search (NAS) and In-Memory Computing (IMC) are emerging as key technologies to overcome current bandwidth limits. While we are still far from Artificial General Intelligence, the synergy of spatial architectures and algorithmic compression is paving the way for ubiquitous, energy-efficient AI.
Summary updated