Introduction
Dynamic Vision Sensors (DVS), often called "event cameras," represent a paradigm shift in visual sensing. Unlike traditional cameras that capture full frames at fixed intervals, DVS pixels only report changes in light intensity. This event-driven nature makes them inherently low-power and high-speed. However, deploying these sensors in long-term IoT monitoring scenarios (using networks like LoRa or NB-IoT) remains challenging because raw event streams can still exceed available bandwidth. Furthermore, streaming raw visual data to central servers poses significant privacy risks.
This paper proposes a two-stage pipeline that addresses these issues by performing aggressive near-chip filtering followed by efficient edge-based detection. This "middle-ground" architecture compresses data near the sensor while maintaining enough information for accurate pedestrian identification.
System Architecture and Methods
The proposed architecture splits processing between a near-chip FPGA and a low-power microcontroller.
1. The Filtering Module (FPGA)
The filtering module resides close to the sensor and processes raw DVS events through four distinct stages to minimize data volume:
- Event Parsing: Converts raw sensor signals into spatial coordinates and polarity $(x, y, p)$.
- Coincidence Detection: Acts as a spatial-temporal denoiser, discarding events that do not have neighbors in close proximity, effectively removing background noise.
- Aggregation & Subsampling: Events are grouped over time, and an $8 \times 8$ max-pooling operation reduces spatial resolution. This step is intentionally lossy to preserve privacy; the resulting "blobs" are sufficient for AI detection but insufficient for human identification.
- Huffman Coding: Applies lossless compression to the resulting data packets before transmission.
2. The Detection Module (Microcontroller)
The compressed data is transmitted to an STM32F4 microcontroller running a Binary Neural Network (BNN). BNNs are ideal for edge devices because they use weights of +1 or -1, allowing complex multiplications to be replaced by fast bitwise operations (AND and popcount). This hardware-software co-design enables real-time inference on extremely constrained hardware, achieving a detection every 450ms.
Key Results
The system was evaluated using a custom DVS dataset recorded in an office environment. The results demonstrate a significant breakthrough in balancing performance, power, and bandwidth.
- Bandwidth Efficiency: The pipeline reduces the raw event stream from 22.25 Mb/s to a filtered bandwidth of just 74.58 kb/s—a total reduction of 99.6%. This makes the stream compatible with low-power wide-area network (LPWAN) standards.
- Detection Accuracy: Despite the aggressive compression, the BNN maintains an 83% F1 score for pedestrian detection.
- Power Consumption: The entire near-chip architecture consumes only 91.1 mW, making it orders of magnitude more efficient than traditional GPU or high-end CPU-based edge AI solutions which can consume upwards of 3W.
Conclusion and Future Work
By combining near-chip FPGA filtering with edge-based BNNs, this research provides a viable path for privacy-preserving, low-power IoT surveillance. While the current 450ms inference latency is suitable for office monitoring, future iterations could target higher-speed applications by integrating the components into a single Application Specific Integrated Circuit (ASIC). This work highlights the power of hardware-software co-design in overcoming the bandwidth and privacy barriers of modern visual sensing.
Summary updated