Abstract
Deep Neural Network (DNN) acceleration on FPGAs often faces a trade-off between the flexibility of generic reusable architectures and the efficiency of layer-wise pipelined designs. DNNExplorer introduces a hybrid paradigm that combines these approaches: dedicated pipeline stages handle early layers with high characteristic variance, while a generic reusable engine processes the more uniform later layers. This hybrid structure is optimized via a two-level automated design space exploration (DSE) engine. Experimental results demonstrate that DNNExplorer achieves up to 4.2x higher throughput than state-of-the-art pipelined designs and 4.4x better DSP efficiency than industry-standard generic accelerators, all while maintaining high analytical modeling accuracy with less than 2% error.
Introduction
The deployment of Deep Neural Networks (DNNs) on FPGAs typically follows two distinct philosophies. Generic reusable architectures use a single compute engine for all layers, offering scalability but suffering from efficiency losses due to varying computation-to-communication (CTC) ratios across layers. Conversely, layer-wise pipelined architectures assign dedicated hardware to each layer, which is efficient for shallow networks but leads to resource fragmentation and performance degradation in deeper models (e.g., 50+ layers).
DNNExplorer addresses this by exploiting a key observation: early DNN layers exhibit significantly higher variance in their characteristics (CTC ratios) compared to later layers. Quantitatively, the variance in the first half of a DNN is, on average, 1806 times higher than in the second half. By splitting the network, DNNExplorer creates a balanced accelerator that leverages the strengths of both design styles.
The Hybrid Paradigm
The core of DNNExplorer is a hybrid architecture that splits the network at a specific point ($SP$). Layers before $SP$ are processed by a dedicated pipeline (P-stages), where each layer has optimized hardware. Layers after $SP$ are handled by a generic reusable engine (G-struc) that cycles through the remaining layers. To further enhance performance, the system employs fine-grained pipelining with column-based buffering, allowing stages to begin processing before a full image is completed, which significantly reduces initial latency.
Automated Design Space Exploration
To automate the creation of these accelerators, the DNNExplorer framework follows a three-phase process:
- Model/HW Analysis: Profiling the target DNN to extract layer-wise requirements and identifying hardware resource boundaries.
- Accelerator Modeling: Utilizing high-accuracy analytical models to predict resource consumption and performance.
- Architecture Exploration: A two-level search strategy. A Global Optimizer uses Particle Swarm Optimization (PSO) to determine the optimal split point, batch size, and resource distribution. Subsequently, Local Optimizers fine-tune hardware parameters (such as parallelism factors) for the pipeline and generic units to ensure the latency of the pipeline stages matches the latency of the generic engine ($L_p \approx L_g$).
Evaluation and Key Results
The framework was evaluated using a variety of benchmarks, including VGG, ResNet, and MobileNet, on Xilinx ZC706, KU115, and VU9P FPGAs.
Key findings include:
- Throughput Scalability: For deep 38-layer networks, DNNExplorer delivered 4.2x higher throughput than pure pipelined tools like DNNBuilder by avoiding resource fragmentation.
- DSP Efficiency: DNNExplorer improved DSP efficiency by up to 4.4x compared to the Xilinx DPU for small-to-medium input resolutions.
- Modeling Accuracy: The analytical models showed an average error of only 1.15% compared to physical board measurements.
- Fast Exploration: The DSE engine identifies optimal configurations within minutes on a standard desktop CPU.
Conclusion
DNNExplorer successfully bridges the gap between generic and pipelined FPGA accelerators. By intelligently partitioning DNN layers based on their computational characteristics, it provides a scalable and highly efficient solution for modern deep learning workloads. Future work may extend this framework to support more diverse layer types, such as those found in Transformer architectures, and incorporate explicit power-aware optimization.
Summary updated