Presented at ICCAD '24, this research introduces a novel architectural solution to a critical bottleneck in neuromorphic vision: denoising high-resolution event streams without prohibitive memory costs. The proposed Cache-Like Filter (CLF) achieves the accuracy of memory-intensive filters while maintaining a slim $O(m+n)$ space complexity.
The Background Activity Noise Problem
Dynamic Vision Sensors (DVS), or "event cameras," differ from traditional frame-based cameras by reporting only changes in light intensity per pixel. This allows for microsecond temporal resolution and a massive dynamic range. However, DVS data is often contaminated by Background Activity (BA) noise—spurious events triggered by thermal noise or leakage currents rather than actual motion.
To clean this data, spatiotemporal filters check if an incoming event has "neighbors" in both space and time. Traditional "gold-standard" filters (like BAF or STCF) require $O(mn)$ memory—one slot for every pixel on the sensor. For a 1080p sensor, this requires megabytes of expensive on-chip RAM. Existing "lightweight" filters reduce this to $O(m+n)$ by sharing memory across rows and columns, but they often suffer from "collisions" where multiple moving objects overwrite each other's history, leading to poor performance in complex scenes.
The Cache-Like Filter (CLF) Architecture
The authors bridge the gap between efficiency and accuracy using three key innovations:
- Cache-Like Memory Architecture: Instead of storing just one timestamp per row or column, the CLF uses set-associative memory banks. Much like a CPU cache, each row/column index maps to a "block" that can store multiple recent events ($s_{RM}$). This allows the filter to track multiple objects in the same row or column simultaneously, drastically reducing collisions.
- Two-Stage Pipeline with Read Cancellation: Denoising requires checking neighboring rows and columns. The hardware first checks the event's own row/column; if a correlation is found immediately, it cancels subsequent memory reads for neighbors, cutting dynamic power consumption.
- Timestamp Bitwidth Reduction: Mathematical analysis of Poisson-distributed noise revealed that full 32-bit timestamps are unnecessary. Reducing the timestamp to 8 bits slashes the memory footprint by 40% with negligible impact on the false-positive rate.
Evaluation and Results
The design was evaluated using the v2e simulator for synthetic ground truth and real-world data from a CeleX-V 1MP DVS camera.
- Hardware Efficiency: Implemented on a Xilinx Artix-7 FPGA, the CLF consumed between 125mW and 210mW at 100MHz, making it suitable for edge deployment.
- Accuracy: In complex human-centric scenes (gemma-hpe), the CLF achieved an accuracy of 82.68%, significantly outperforming the previous state-of-the-art $O(m+n)$ filter (RCF), which reached only 75.83%.
- Downstream Impact: On real-world highway traffic data, denoising with CLF improved the Mean Average Precision (mAP) of a pre-trained YOLOv5 car detector from 0.453 (using RCF) to 0.486, even slightly exceeding the performance of the memory-intensive BAF (0.476).
Conclusion
The Cache-Like Filter demonstrates that denoising is essentially a search problem in a sparse stream. By treating row/column memory as a cache, the system exploits spatial locality without requiring a 1:1 mapping of memory to pixels. This architecture paves the way for "self-denoising" sensors where such filters are integrated directly into the DVS readout circuitry.
Summary updated