TL;DR
WedNet is a novel window-based denoising network that processes stacks of events simultaneously rather than one-by-one. By combining a solid theoretical foundation in spatiotemporal probability with a multi-scale architecture, it achieves state-of-the-art denoising accuracy while running up to 20 times faster than previous deep learning approaches.
What Problem Does This Paper Address?
Event cameras (like DVS or DAVIS) are bio-inspired sensors that only report changes in light intensity, offering microsecond-level resolution and massive dynamic range. However, they suffer from significant "Background Activity" (BA) noise caused by hardware imperfections like thermal noise and leakage currents.
Existing solutions fall into two camps:
- Traditional Filters: Fast but struggle in high-noise environments because they rely on simple, hand-tuned thresholds.
- Deep Learning Methods: Highly accurate but computationally expensive. Most are "element-based," meaning they process the event stream one event at a time, making them too slow for the very real-time applications event cameras were designed for.
Furthermore, many deep learning models act as "black boxes," lacking the interpretability needed for researchers to understand why a specific architecture works.
Methods
The authors introduce WedNet, which shifts the paradigm from element-based to window-based processing. Instead of labeling one event, the network processes a "stack" of events within a temporal window.
Key components include:
- Theoretical Grounding: The authors model noise as a Poisson distribution and real events as a Discrete Gaussian distribution. This math informs the design of the modules.
- Temporal Window (TW) Module: Uses an adaptive threshold based on event density to filter out temporal outliers.
- Soft Spatial Feature Embedding (SSFE): This is the core "interpretable" module. It is derived from Maximum A Posteriori (MAP) estimation and Convolutional Sparse Coding (CSC). It uses 1D convolutions along the event direction to extract spatial features.
- Hierarchical Spatial Feature Learning (HSFL): To handle the irregular, point-cloud-like nature of event data, the network uses a multi-scale approach (sampling, grouping, and abstraction) across four levels to capture both fine and coarse spatial details.
- Bone Events Check (BEC): A preprocessing step that uses Connected Domain Labeling (CDL) to identify structural "bone" events. This ensures that the sampling process focuses on the actual objects in motion rather than isolated noise points.
Datasets
The authors evaluated WedNet on three major benchmarks:
- DVSCLEAN: A dataset containing both simulated and real-world data with varying noise ratios (50% and 100%).
- DVSNOISE20: Features 16 indoor and outdoor scenes. It uses an Event Probability Mask (EPM) derived from active pixel sensors (APS) as a ground truth for denoising.
- ED-KoGTL: A dataset recorded via a robot arm under four different illumination levels (from a bright 750lux to a very dim 5lux), testing the model's robustness in low-light conditions.
Key Insights and Ideas
- Windowing is the Speed King: By processing events in batches, WedNet bypasses the bottleneck of individual event processing, which is the primary reason it achieves such high throughput.
- Math-to-Architecture Pipeline: Rather than just stacking layers, the authors used the mathematical properties of sparse coding to design the SSFE module. This provides a clear "why" behind the network's ability to separate signal from noise.
- The "Bone" Concept: Noise in event cameras is often isolated. By identifying "bone" events (events that are part of a larger connected structure), the network can effectively ignore noise during the expensive feature extraction phases.
- Multi-Scale Matters: Because event data lacks a fixed frame/grid, using a hierarchical structure (similar to PointNet++) allows the model to understand the geometry of moving objects at different resolutions.
Key Results
- Speed: WedNet is significantly faster than previous SOTA deep learning models. It is roughly 20x faster than AEDNet and even outperforms some traditional filter-based methods in terms of running time on specific datasets.
- Denoising Quality: On the DVSCLEAN dataset, WedNet achieved the highest average Signal-to-Noise Ratio (SNR) of 25.73 dB, outperforming all competitors.
- Low-Light Performance: In the 5lux (low-light) scene of the ED-KoGTL dataset, WedNet improved the SNR by 13.6 dB over the raw data, proving its effectiveness in challenging environments where noise usually overwhelms the signal.
- Downstream Impact: The authors demonstrated that using WedNet as a preprocessor improves the accuracy of object classification tasks (using the HATS algorithm) by significant margins (e.g., a 7.7% absolute increase on CIFAR10-DVS).
Limitations and Future Work
While WedNet is a major step toward real-time deep denoising, the paper notes a few areas for growth:
- Hardware Requirements: Although fast, the benchmarks were performed on a high-end desktop GPU (RTX 3090 Ti). Performance on ultra-low-power edge devices (where many event cameras live) remains to be seen.
- End-to-End Integration: The current work treats denoising as a standalone preprocessing step. Future work could involve integrating this window-based approach directly into end-to-end pipelines for SLAM or high-speed tracking to see if further efficiencies can be gained.
Summary updated