Diffusion-Based Particle-DETR for BEV Perception

Asen Nachkov, Danda Pani Paudel, Martin Danelljan, Luc Van Gool

Abstract

Particle-DETR is a novel 3D object detection framework that integrates diffusion models into Bird’s-Eye-View (BEV) perception. By treating object queries as "particles" that denoise toward ground-truth locations, the framework achieves state-of-the-art results on the NuScenes dataset. It effectively bridges the gap between generative diffusion and high-precision deterministic detectors, outperforming previous stochastic models like DiffusionDet and rivaling top-tier architectures such as BEVFormer.

Introduction

In autonomous driving, the Bird’s-Eye-View (BEV) representation is essential for providing a metrically accurate, top-down map of the environment. However, detecting small objects like pedestrians across vast spatial areas remains challenging due to sensor uncertainty and data sparsity. While diffusion models excel at modeling such uncertainty, their application to BEV perception has historically struggled. Standard diffusion-based detectors often fail to maintain the fixed spatial relationships that modern Detection Transformer (DETR) architectures rely on, as random noise during training disrupts learned positional priors.

Particle-DETR addresses these challenges by treating object queries as a swarm of particles. These particles communicate via self-attention to robustly understand the global scene, deciding which particles should converge on specific object locations.

Methodology

Particle-DETR introduces several key innovations to harmonize generative diffusion with high-precision 3D detection.

Overview of the Particle-DETR architecture, highlighting the BEV encoder, query interpolation, and the denoising detector.

Diffusion in BEV

The framework begins with random "reference points" (noise) and iteratively refines them into the centers of real objects through a denoising process. This allows the model to explore the scene and converge on high-probability object locations.

Object Query Interpolation

A core technical breakthrough is the query interpolation module. Standard DETR models use fixed queries, which are incompatible with the moving particles in a diffusion process. Particle-DETR instead learns a spatial grid of queries and uses bilinear interpolation to "pick up" the appropriate query based on a particle's current noisy location, allowing the model to maintain spatial awareness despite stochasticity.

Training and Post-Processing

The model employs Many-to-One Matching during training, where multiple predicted particles are matched to a single ground-truth object. This teaches the model to create "basins of attraction" around objects. For inference, a specialized Radial Suppression step is used, which outperforms standard Non-Maximum Suppression (NMS) for small, non-overlapping objects like traffic cones.

Experimental Results

The model was evaluated on the NuScenes dataset, which includes 1.4 million 3D bounding boxes across 1,000 urban scenes.

Performance comparison on the NuScenes dataset. Particle-DETR (ours) achieves a superior NuScenes Detection Score (NDS) compared to BEVFormer and DiffusionDet.

Key Performance Metrics

Particle-DETR achieved a NuScenes Detection Score (NDS) of 0.5298, surpassing the deterministic BEVFormer (0.5168) and significantly outperforming the baseline DiffusionDet (0.4580). The model showed particular strength in estimating object dimensions, orientation, and velocity.

Calibration and Robustness

Sparsification analysis indicates that Particle-DETR is better calibrated than previous models; its confidence scores are more reliably correlated with actual detection accuracy. Furthermore, the use of DDIM steps allows for a flexible compute-vs-accuracy trade-off, where more denoising steps can be used at inference for higher precision without retraining.

Conclusion and Future Work

While Particle-DETR sets a new benchmark for NDS, a small gap remains in Mean Average Precision (mAP) compared to the best deterministic models, likely due to the inherent variance of stochastic inputs. Future work will focus on improving uncertainty quantification beyond simple particle counting and integrating temporal features to enhance scene understanding over time.

Summary updated

Abstract

The Bird-Eye-View (BEV) is one of the most widely-used scene representations for visual perception in Autonomous Vehicles (AVs) due to its well suited compatibility to downstream tasks. For the enhanced safety of AVs, modeling perception uncertainty in BEV is crucial. Recent diffusion-based methods offer a promising approach to uncertainty modeling for visual perception but fail to effectively detect small objects in the large coverage of the BEV. Such degradation of performance can be attributed primarily to the specific network architectures and the matching strategy used when training. Here, we address this problem by combining the diffusion paradigm with current state-of-the-art 3D object detectors in BEV. We analyze the unique challenges of this approach, which do not exist with deterministic detectors, and present a simple technique based on object query interpolation that allows the model to learn positional dependencies even in the presence of the diffusion noise. Based on this, we present a diffusion-based DETR model for object detection that bears similarities to particle methods. Abundant experimentation on the NuScenes dataset shows equal or better performance for our generative approach, compared to deterministic state-of-the-art methods. Our source code will be made publicly available.
BibTeX
@inproceedings{nachkov2025diffusion,
  author = {Asen Nachkov and Danda Pani Paudel and Martin Danelljan and Luc Van Gool},
  title = {{Diffusion-Based Particle-DETR for BEV Perception}},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
  year = {2025},
  doi = {10.1109/wacv61041.2025.00270},
}
Figure