Abstract
Deploying deep neural networks (DNNs) on edge devices like FPGAs requires a delicate balance between accuracy and hardware efficiency. Traditional optimization techniques often treat neural architecture search (NAS), quantization, and hardware design as isolated tasks, leading to sub-optimal performance due to reliance on hardware-agnostic metrics like FLOPs. This paper presents HAO (Hardware-aware neural Architecture Optimization), a co-design framework that jointly optimizes these three domains. By formulating the search as an integer programming problem to prune the design space, HAO achieves state-of-the-art results, reaching 72.5% Top-1 accuracy on ImageNet while significantly outperforming existing NAS models in inference speed.
Introduction
The deployment of DNNs on resource-constrained hardware is often hindered by the "gap" between theoretical efficiency and actual hardware performance. A model optimized for low FLOPs (Floating Point Operations) may not necessarily translate to low latency on an FPGA if its operations do not align well with the underlying logic cells (LUTs) or digital signal processors (DSPs). HAO addresses this by treating hardware configuration, quantization, and architecture as a unified optimization problem, ensuring that the resulting models are inherently hardware-friendly.
Methodology
HAO utilizes a two-stage search process to navigate the mathematically intractable joint search space. First, it employs Integer Programming (IP) to prune the space by finding feasible hardware and quantization configurations that satisfy latency constraints while minimizing accuracy perturbation. Second, it uses Monte Carlo Tree Search (MCTS) combined with a Support Vector Regression (SVR) accuracy predictor to explore the architecture space efficiently.
A key innovation is the use of hardware subgraph templates, which map groups of operations (e.g., 1x1 and 3x3 convolutions) directly to FPGA resources. This allows for precise analytical modeling of LUT, DSP, and BRAM consumption. Furthermore, HAO supports mixed-precision quantization, allowing different layers to utilize varying bit-widths (2-8 bits) to maximize throughput and perfectly saturate the FPGA's DSP resources.
Results and Discussion
The framework was evaluated on the ImageNet (ILSVRC2012) dataset using a Xilinx Zynq (ZU3EG) FPGA. HAO demonstrated a superior latency-accuracy Pareto frontier compared to both manually designed models like MobileNetV2 and NAS-generated models like MnasNet.
Key performance metrics include:
- High Efficiency: HAO achieved a throughput of 50 FPS, making it 60% faster than MnasNet and 135% faster than FBNet at comparable accuracy levels.
- Optimal Resource Usage: The final design utilized 100% of available DSPs and 87% of LUTs, demonstrating extremely high hardware efficiency.
- Mixed-Precision Benefits: The search identified that specific mixed-precision settings (e.g., 6-bit/7-bit) often provide a better trade-off than uniform 8-bit quantization, providing higher throughput without significant accuracy loss.
Conclusion
HAO demonstrates that joint optimization of hardware and software is essential for maximizing edge AI performance. By moving beyond hardware-agnostic metrics and leveraging integer programming for efficient space pruning, HAO provides a scalable path for deploying high-performance DNNs on FPGAs. Future work involves extending the subgraph modeling to more complex tasks like object detection and adapting the IP constraints for other hardware backends such as ASICs or specialized AI accelerators.
Summary updated