Abstract: Pixel-parallel processing elements (PE) and SIMD architectures are widely used in high-speed image processing to enhance computing power. By fully exploiting data-level parallelism in low- and middle-level image processing, SIMD architectures can complete vast amounts of computation with fewer instruction cycles, meeting high-speed system requirements. This paper describes a high-speed vision system featuring a superscalar PE to enhance performance, alongside a dedicated parallel computing language. The system achieves motion detection at over 2,000 fps and face detection at over 100 fps, significantly outperforming general serial CPUs in similar applications.
TL;DR
Researchers have developed a high-speed vision system that reaches a staggering 2,000 frames per second (fps) for motion detection. By implementing a "superscalar" Processing Element (PE) architecture that decouples image data transfer from computation, the system achieves a near 100% performance boost over previous designs, effectively eliminating the "stall" time usually required for sensor exposure and data loading.
The Bottleneck in High-Speed Vision
In high-speed machine vision, the primary bottleneck is often the logistics of moving data rather than raw mathematical computation. Traditional SIMD (Single Instruction, Multiple Data) architectures frequently suffer from a "stop-and-wait" problem: processing elements (PEs) must pause their calculations while the next frame of image data is transferred from the sensor.
This I/O load creates significant overhead. Even with a massive array of parallel processors, they remain idle during the sensor's exposure and data transfer cycles. The researchers sought to bridge this gap by creating a hardware architecture capable of simultaneous processing and data loading—effectively "walking and chewing gum at the same time."
Methods: The Superscalar Innovation
The proposed system utilizes a three-tier heterogeneous architecture implemented on an Altera Cyclone III FPGA:
- Pixel-Parallel PE Array ($64 \times 64$): Handles low-level image processing (e.g., filtering, edge detection). Each PE is a 1-bit processor connected to its four nearest neighbors.
- Row-Parallel Processors: Manage middle-level processing and act as a bridge between the PE array and the main controller.
- RISC Core: A general-purpose processor for high-level logic and system management.
The "superscalar" innovation refers to the PE's ability to execute an image transfer instruction and a processing instruction simultaneously. This is achieved by adding a dedicated data path with a channel controller and a separate data bank to each PE. This creates a hardware pipeline where the exposure and transfer of frame $N+1$ occur concurrently with the processing of frame $N$.
To support this hardware, the team developed a custom parallel computing language, compiler, and assembler, allowing developers to write high-level code that translates into synchronized SIMD instructions.
Key Results and Performance
The system was evaluated using real-time data from a high-speed commercial camera. Key performance metrics include:
- Motion Detection: Achieved a processing rate of 2,000 fps at $256 \times 256$ resolution.
- Face Detection: Achieved 104 fps, significantly outperforming general-purpose serial CPUs.
- Computational Power: Operating at 100MHz, the system provides 44 GOPS (Giga-Operations Per Second) for 8-bit additions, with an internal PE array throughput of 50 GB/s.
For specific tasks like background reduction, the superscalar architecture reduced processing time from 7μs to just 0.5μs—a 14x improvement.
Key Insights
- Pipelining Efficiency: The most significant gain comes from eliminating PE array idle time. Adding a channel controller effectively doubled throughput without increasing clock speed.
- Heterogeneous Hierarchy: Matching the processor to the task is crucial. 1-bit PEs excel at pixel-level math, while a RISC core is better suited for complex logic.
- Software-Hardware Co-design: A custom compiler ensures the complex parallel hardware remains programmable and adaptable to future instruction encoding changes.
Limitations and Future Work
The current implementation uses a $64 \times 64$ PE array due to FPGA logic constraints. While sufficient for a prototype, scaling to higher resolutions like HD or 4K would require significantly more resources and present power management challenges. Additionally, while the system excels at classical vision algorithms (median filters, background subtraction), future work will focus on adapting this superscalar SIMD approach for modern deep learning and CNN inference.
Summary updated