An O(m+n)-Space Spatiotemporal Denoising Filter with Cache-Like Memories for Dynamic Vision Sensors

Qinghang Zhao, Jiaqi Wang, Yixi Ji, Jinjian Wu, Guangming Shi

Presented at ICCAD '24, this research introduces a novel architectural solution to a critical bottleneck in neuromorphic vision: denoising high-resolution event streams without prohibitive memory costs. The proposed Cache-Like Filter (CLF) achieves the accuracy of memory-intensive filters while maintaining a slim $O(m+n)$ space complexity.

The Background Activity Noise Problem

Dynamic Vision Sensors (DVS), or "event cameras," differ from traditional frame-based cameras by reporting only changes in light intensity per pixel. This allows for microsecond temporal resolution and a massive dynamic range. However, DVS data is often contaminated by Background Activity (BA) noise—spurious events triggered by thermal noise or leakage currents rather than actual motion.

DVS pixel architecture and event generation mechanism based on log-intensity changes.

To clean this data, spatiotemporal filters check if an incoming event has "neighbors" in both space and time. Traditional "gold-standard" filters (like BAF or STCF) require $O(mn)$ memory—one slot for every pixel on the sensor. For a 1080p sensor, this requires megabytes of expensive on-chip RAM. Existing "lightweight" filters reduce this to $O(m+n)$ by sharing memory across rows and columns, but they often suffer from "collisions" where multiple moving objects overwrite each other's history, leading to poor performance in complex scenes.

The Cache-Like Filter (CLF) Architecture

The authors bridge the gap between efficiency and accuracy using three key innovations:

  1. Cache-Like Memory Architecture: Instead of storing just one timestamp per row or column, the CLF uses set-associative memory banks. Much like a CPU cache, each row/column index maps to a "block" that can store multiple recent events ($s_{RM}$). This allows the filter to track multiple objects in the same row or column simultaneously, drastically reducing collisions.
  2. Two-Stage Pipeline with Read Cancellation: Denoising requires checking neighboring rows and columns. The hardware first checks the event's own row/column; if a correlation is found immediately, it cancels subsequent memory reads for neighbors, cutting dynamic power consumption.
  3. Timestamp Bitwidth Reduction: Mathematical analysis of Poisson-distributed noise revealed that full 32-bit timestamps are unnecessary. Reducing the timestamp to 8 bits slashes the memory footprint by 40% with negligible impact on the false-positive rate.
Comparison of memory architectures for spatiotemporal filtering, highlighting the proposed O(m+n) cache-like memory (d) against standard O(mn) (b) and O(m+n) (c) approaches.

Evaluation and Results

The design was evaluated using the v2e simulator for synthetic ground truth and real-world data from a CeleX-V 1MP DVS camera.

  • Hardware Efficiency: Implemented on a Xilinx Artix-7 FPGA, the CLF consumed between 125mW and 210mW at 100MHz, making it suitable for edge deployment.
  • Accuracy: In complex human-centric scenes (gemma-hpe), the CLF achieved an accuracy of 82.68%, significantly outperforming the previous state-of-the-art $O(m+n)$ filter (RCF), which reached only 75.83%.
  • Downstream Impact: On real-world highway traffic data, denoising with CLF improved the Mean Average Precision (mAP) of a pre-trained YOLOv5 car detector from 0.453 (using RCF) to 0.486, even slightly exceeding the performance of the memory-intensive BAF (0.476).

Conclusion

The Cache-Like Filter demonstrates that denoising is essentially a search problem in a sparse stream. By treating row/column memory as a cache, the system exploits spatial locality without requiring a 1:1 mapping of memory to pixels. This architecture paves the way for "self-denoising" sensors where such filters are integrated directly into the DVS readout circuitry.

Summary updated

BibTeX
@inproceedings{zhao2024o,
  author = {Qinghang Zhao and Jiaqi Wang and Yixi Ji and Jinjian Wu and Guangming Shi},
  title = {{An O(m+n)-Space Spatiotemporal Denoising Filter with Cache-Like Memories for Dynamic Vision Sensors}},
  booktitle = {ICCAD},
  year = {2024},
  doi = {10.1145/3676536.3676710},
}
Figure