WD-DETR: Wavelet Denoising-Enhanced Real-Time Object Detection Transformer for Robot Perception with Event Cameras

Yangjie Cui, Boyang Gao, Yiwei Zhang, Xin Dong, Jinwu Xiang, Daochun Li, Zhan TuarXiv 2025 · DOI · arXiv

Introduction

Traditional RGB cameras often fail in high-speed scenarios or extreme lighting conditions due to motion blur and limited dynamic range. Event cameras offer a compelling alternative by asynchronously detecting brightness changes at each pixel, providing low latency and high dynamic range. However, event data is inherently sparse and noisy. To leverage modern deep learning, researchers typically convert these events into dense representations, but this process often accumulates noise that degrades feature quality and leads to missed detections.

WD-DETR addresses these challenges by introducing a novel real-time object detection framework designed specifically for event cameras. It combines wavelet-based denoising with an optimized Detection Transformer (DETR) architecture to achieve high accuracy and real-time performance on edge computing hardware.

Architectural Innovations

WD-DETR introduces several key components to handle the unique noise profile of event data while maintaining computational efficiency for robotic platforms.

The WD-DETR architecture features a Wavelet-Enhanced Backbone, a Hybrid Encoder with Dynamic Reorganization Conv Blocks (DRCB), and an optimized DETR Decoder.

Wavelet-Enhanced Backbone

Instead of standard max-pooling or strided convolutions, the network utilizes Wavelet Pooling. This mechanism applies a Discrete Wavelet Transform (DWT) to decompose features into low-frequency (structural) and high-frequency (noise-rich) components. By discarding high-frequency sub-bands and applying the Inverse DWT only to the low-frequency parts, the network performs denoising and downsampling simultaneously, ensuring the model learns from cleaner data without increasing parameter counts.

Efficient Feature Fusion

To speed up the Hybrid Encoder, the authors replaced standard convolutions with the Dynamic Reorganization Conv Block (DRCB). Inspired by ShuffleNet, this block uses group convolutions and channel shuffling to reduce parameters while maintaining feature diversity. Additionally, events are accumulated using a Time-Decay Representation, employing an exponentially weighted polynomial decay function to prioritize recent events and reduce "ghosting" artifacts.

Optimized DETR Head

The model employs an IoU-aware query selection and a transformer-based decoder. This design allows for precise bounding box predictions without the need for traditional, hand-crafted post-processing like Non-Maximum Suppression (NMS), which can be a bottleneck in real-time systems.

Experimental Results and Robustness

WD-DETR was evaluated on major event-based benchmarks, including DSEC, Gen1 Automotive Detection, and the 1Mpx dataset. It achieved state-of-the-art accuracy with a mean Average Precision (mAP) of 52.8% on Gen1 and 65.5% on 1Mpx, outperforming previous models like ERGO-12 and RVT-B.

Comparison between (a) a dark RGB frame, (b) WD-DETR prediction at 35 FPS, and (c) the denoised event representation.

One of the most significant advantages of WD-DETR is its robustness in low-light environments. As shown in the comparison above, while standard RGB frames become almost entirely black in low-light conditions (e.g., 10 Lux), the dense representation reconstructed from event data remains clear enough for accurate detection.

On an NVIDIA Jetson Orin NX, a common edge computer for robotics, the model achieved a consistent 35 FPS using TensorRT FP16 precision. Ablation studies confirmed that the Wavelet Pooling block is critical; its removal led to a significant drop in mAP, proving that integrated denoising is essential for processing noisy event streams effectively.

Conclusion

WD-DETR represents a significant step forward for event-based vision in robotics. By treating denoising as an integral part of the downsampling process and optimizing the transformer architecture for edge hardware, it provides a robust, real-time solution for perception in challenging environments where traditional cameras fail.

Summary updated

Abstract

Previous studies on event camera sensing have demonstrated certain detection performance using dense event representations. However, the accumulated noise in such dense representations has received insufficient attention, which degrades the representation quality and increases the likelihood of missed detections. To address this challenge, we propose the Wavelet Denoising-enhanced DEtection TRansformer, i.e., WD-DETR network, for event cameras. In particular, a dense event representation is presented first, which enables real-time reconstruction of events as tensors. Then, a wavelet transform method is designed to filter noise in the event representations. Such a method is integrated into the backbone for feature extraction. The extracted features are subsequently fed into a transformer-based network for object prediction. To further reduce inference time, we incorporate the Dynamic Reorganization Convolution Block (DRCB) as a fusion module within the hybrid encoder. The proposed method has been evaluated on three event-based object detection datasets, i.e., DSEC, Gen1, and 1Mpx. The results demonstrate that WD-DETR outperforms tested state-of-the-art methods. Additionally, we implement our approach on a common onboard computer for robots, the NVIDIA Jetson Orin NX, achieving a high frame rate of approximately 35 FPS using TensorRT FP16, which is exceptionally well-suited for real-time perception of onboard robotic systems.
BibTeX
@misc{cui2025wd,
  author = {Yangjie Cui and Boyang Gao and Yiwei Zhang and Xin Dong and Jinwu Xiang and Daochun Li and Zhan Tu},
  title = {{WD-DETR: Wavelet Denoising-Enhanced Real-Time Object Detection Transformer for Robot Perception with Event Cameras}},
  year = {2025},
  eprint = {2506.09098},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2506.09098},
}
Figure