Introduction
Traditional RGB cameras often fail in high-speed scenarios or extreme lighting conditions due to motion blur and limited dynamic range. Event cameras offer a compelling alternative by asynchronously detecting brightness changes at each pixel, providing low latency and high dynamic range. However, event data is inherently sparse and noisy. To leverage modern deep learning, researchers typically convert these events into dense representations, but this process often accumulates noise that degrades feature quality and leads to missed detections.
WD-DETR addresses these challenges by introducing a novel real-time object detection framework designed specifically for event cameras. It combines wavelet-based denoising with an optimized Detection Transformer (DETR) architecture to achieve high accuracy and real-time performance on edge computing hardware.
Architectural Innovations
WD-DETR introduces several key components to handle the unique noise profile of event data while maintaining computational efficiency for robotic platforms.
Wavelet-Enhanced Backbone
Instead of standard max-pooling or strided convolutions, the network utilizes Wavelet Pooling. This mechanism applies a Discrete Wavelet Transform (DWT) to decompose features into low-frequency (structural) and high-frequency (noise-rich) components. By discarding high-frequency sub-bands and applying the Inverse DWT only to the low-frequency parts, the network performs denoising and downsampling simultaneously, ensuring the model learns from cleaner data without increasing parameter counts.
Efficient Feature Fusion
To speed up the Hybrid Encoder, the authors replaced standard convolutions with the Dynamic Reorganization Conv Block (DRCB). Inspired by ShuffleNet, this block uses group convolutions and channel shuffling to reduce parameters while maintaining feature diversity. Additionally, events are accumulated using a Time-Decay Representation, employing an exponentially weighted polynomial decay function to prioritize recent events and reduce "ghosting" artifacts.
Optimized DETR Head
The model employs an IoU-aware query selection and a transformer-based decoder. This design allows for precise bounding box predictions without the need for traditional, hand-crafted post-processing like Non-Maximum Suppression (NMS), which can be a bottleneck in real-time systems.
Experimental Results and Robustness
WD-DETR was evaluated on major event-based benchmarks, including DSEC, Gen1 Automotive Detection, and the 1Mpx dataset. It achieved state-of-the-art accuracy with a mean Average Precision (mAP) of 52.8% on Gen1 and 65.5% on 1Mpx, outperforming previous models like ERGO-12 and RVT-B.
One of the most significant advantages of WD-DETR is its robustness in low-light environments. As shown in the comparison above, while standard RGB frames become almost entirely black in low-light conditions (e.g., 10 Lux), the dense representation reconstructed from event data remains clear enough for accurate detection.
On an NVIDIA Jetson Orin NX, a common edge computer for robotics, the model achieved a consistent 35 FPS using TensorRT FP16 precision. Ablation studies confirmed that the Wavelet Pooling block is critical; its removal led to a significant drop in mAP, proving that integrated denoising is essential for processing noisy event streams effectively.
Conclusion
WD-DETR represents a significant step forward for event-based vision in robotics. By treating denoising as an integral part of the downsampling process and optimizing the transformer architecture for edge hardware, it provides a robust, real-time solution for perception in challenging environments where traditional cameras fail.
Summary updated