Abstract
Previous deep learning-based event denoising methods mostly suffer from poor interpretability and difficulty in real-time processing due to their complex architecture designs. In this paper, we propose window-based event denoising, which simultaneously deals with a stack of events while existing element-based denoising focuses on one event each time. Besides, we give the theoretical analysis based on probability distributions in both temporal and spatial domains to improve interpretability. In the temporal domain, we use timestamp deviations between processing events and the central event to judge temporal correlation and filter out temporal-irrelevant events. In the spatial domain, we choose maximum a posteriori (MAP) to discriminate real-world events and noise, and use learned convolutional sparse coding to optimize the objective function. Based on the theoretical analysis, we build the Temporal Window (TW) module and Soft Spatial Feature Embedding (SSFE) module to process temporal and spatial information separately, and construct a novel multi-scale window-based event denoising network, named WedNet (MSDNet). The high denoising accuracy and fast running speed of our network enable us to achieve real-time denoising in complex scenes. Extensive experimental results verify the effectiveness and robustness of our approach.
Introduction
Event cameras, such as Dynamic Vision Sensors (DVS), are bio-inspired sensors that operate fundamentally differently from traditional frame-based cameras. Instead of capturing full images at fixed intervals, they asynchronously report changes in light intensity at each pixel. This results in a stream of events with microsecond-level temporal resolution and a very high dynamic range. However, these sensors are susceptible to "Background Activity" (BA) noise caused by hardware imperfections like thermal noise and leakage currents.
Effective denoising is critical for downstream tasks like object tracking and SLAM. Existing solutions generally fall into two categories: traditional filters, which are fast but struggle in high-noise environments, and deep learning methods, which are accurate but computationally expensive. Most deep learning models are "element-based," processing the event stream one event at a time, which is too slow for real-time applications. Furthermore, many of these models lack interpretability, acting as "black boxes."
Methodology: The Window-Based Paradigm
We introduce WedNet, a novel window-based denoising network that shifts the paradigm from processing single events to processing "stacks" of events within a temporal window. This approach significantly reduces the computational bottleneck associated with individual event labeling.
The architecture is grounded in mathematical theory, modeling noise as a Poisson distribution and real events as a Discrete Gaussian distribution. Key modules include:
- Temporal Window (TW) Module: Filters temporal outliers using an adaptive threshold based on event density.
- Soft Spatial Feature Embedding (SSFE): An interpretable module derived from Maximum A Posteriori (MAP) estimation and Convolutional Sparse Coding (CSC). It uses 1D convolutions along the event direction to extract spatial features.
- Hierarchical Spatial Feature Learning (HSFL): A multi-scale approach that uses sampling, grouping, and abstraction across four levels to capture both fine and coarse spatial details from irregular event data.
- Bone Events Check (BEC): A preprocessing step that identifies structural "bone" events using Connected Domain Labeling (CDL). This ensures that feature extraction focuses on actual objects rather than isolated noise points.
Experimental Evaluation and Results
WedNet was evaluated on three major benchmarks: DVSCLEAN (simulated and real-world data), DVSNOISE20 (indoor/outdoor scenes), and ED-KoGTL (low-light conditions).
Speed and Accuracy
WedNet achieves state-of-the-art performance in both denoising quality and processing speed. On the DVSCLEAN dataset, it reached a Signal-to-Noise Ratio (SNR) of 25.73 dB, outperforming all competitors. Crucially, it is approximately 20 times faster than previous deep learning models like AEDNet, making it suitable for real-time deployment.
Robustness and Downstream Impact
The model shows exceptional robustness in challenging environments. In extreme low-light scenes (5lux), WedNet improved the SNR by 13.6 dB over raw data. Furthermore, using WedNet as a preprocessor significantly enhances downstream performance; for instance, it provided a 7.7% absolute increase in object classification accuracy on the CIFAR10-DVS dataset using the HATS algorithm.
Conclusion
WedNet represents a significant advancement in event camera technology, making deep-learning-based denoising viable for real-time applications. By combining a window-based processing strategy with a theoretically grounded, interpretable architecture, it delivers high accuracy and exceptional speed. Future work will explore the integration of this approach into ultra-low-power edge devices and end-to-end vision pipelines.
Summary updated