Abstract
SATAY is an automated toolflow designed to map the entire YOLO family (v3 to v8) onto FPGA devices using a high-performance, deeply pipelined streaming architecture. By maintaining model weights on-chip and selectively offloading large skip-connection buffers to external memory, SATAY achieves ultra-low latency and superior performance density compared to existing FPGA and edge-GPU solutions. This work presents the first FPGA implementation of YOLOv8, demonstrating significant improvements in the trade-off between accuracy and hardware efficiency.
Introduction
Object detection is a critical component of autonomous systems, with the YOLO (You Only Look Once) family of models being the industry standard. However, the increasing complexity of modern YOLO versions (v5-v8) introduces sophisticated layers, such as SiLU activations and CSP blocks, along with long skip connections that pose significant challenges for deployment on edge hardware. Existing FPGA accelerators often suffer from two main issues: a CPU bottleneck caused by offloading non-convolutional layers, and a lack of versatility due to hand-tuning for specific model versions. SATAY addresses these challenges by providing an automated path to generate hardware that supports diverse operations while maximizing FPGA parallelism.
The SATAY Architecture
SATAY employs a streaming architecture design, where every layer in the network is mapped to a dedicated hardware block. This creates a deep pipeline where data flows continuously through specialized modules, rather than relying on a single general-purpose engine.
Modular Building Blocks
The toolflow utilizes a library of hardware modules for YOLO-specific operations, including Resize, HardSwish, and SPPF (Spatial Pyramid Pooling - Fast). These blocks are composed to form the full detection pipeline on the FPGA fabric.
Memory Management
To overcome the "memory wall," SATAY stores all model weights directly on the FPGA using BRAM or URAM. This eliminates high-latency external memory accesses during inference. However, long skip connections can consume excessive on-chip memory. SATAY uses an automated algorithm to identify the largest buffers and move them to a "Software FIFO" in external DDR memory. This selective off-chip buffering reduces on-chip usage by up to 56% with minimal impact on bandwidth.
Design Space Exploration
An automated Design Space Exploration (DSE) tool uses a greedy algorithm to allocate Digital Signal Processing (DSP) resources. By identifying and parallelizing bottleneck nodes, the toolflow ensures maximum overall throughput for the generated hardware.
Experimental Results
The SATAY toolflow was evaluated using the COCO-val2017 and COCO-128 datasets across various YOLO versions (v3, v5, v8) and Xilinx FPGA platforms (VCU118, VCU110, U250).
Performance and Efficiency
SATAY-generated designs demonstrate significant performance gains, achieving a 79x speedup over an ARM Cortex-A72 CPU and a 3.6x speedup over a Jetson TX2 GPU. For the YOLOv3-tiny model, SATAY achieved 3.4x higher performance density (GOP/s/DSP) compared to previous state-of-the-art FPGA implementations.
State-of-the-Art YOLOv8
This work introduces the first FPGA implementation of YOLOv8 in the literature. The VCU110 variant achieves a 61% mAP at ultra-low latency, defining the current Pareto frontier for high-accuracy, high-performance edge object detection.
Conclusion
SATAY provides a robust, automated solution for deploying the latest YOLO models on FPGAs. By combining a streaming architecture with intelligent memory management, it bridges the gap between model complexity and hardware efficiency. Future work will explore weight binarization and Quantization-Aware Training (QAT) to further reduce memory footprints and expand support to a wider range of edge FPGA devices.
Summary updated