TL;DR
This paper introduces REXO, a novel method for indoor multi-view radar object detection that leverages 3D bounding box diffusion. By extending 2D diffusion models to the 3D radar space and incorporating a ground-level constraint, REXO significantly improves detection accuracy and robustness, outperforming existing state-of-the-art methods on challenging indoor radar datasets.
What Problem Does This Paper Address?
Detecting objects in indoor environments using radar is gaining traction due to its cost-effectiveness and privacy-preserving nature, especially in low-light or adverse conditions. However, existing multi-view radar object detection methods often struggle with ambiguous feature associations across different radar views, leading to degraded performance in complex indoor scenes. These methods typically rely on implicit cross-view associations or query-based attention mechanisms that can be computationally expensive and less effective. The authors aim to address these limitations by proposing a diffusion-based approach that explicitly guides cross-view feature association and leverages prior knowledge about object placement.
Methods
REXO builds upon the concept of DiffusionDet, which reformulates object detection as a generative denoising process. The core innovation of REXO is to lift the diffusion process from 2D bounding boxes to 3D bounding boxes in the radar coordinate system. This allows for:
- Explicit Cross-View Feature Association: Noisy 3D bounding boxes at each diffusion timestep are projected onto each radar view. Region-of-interest (RoI) aligned crops from these views provide features that are explicitly associated, overcoming the ambiguity of implicit methods.
- Radar-Conditioned BBox Denoising: The associated radar features are used to condition the denoising process, making the challenging task of 3D bounding box refinement more robust.
- Ground-Level Constraint: By incorporating prior knowledge that detected objects (people) are in contact with the ground, REXO reduces the number of diffusion parameters and enforces geometric consistency, leading to more realistic detections.
- 3D-to-2D Projection and Refinement: The predicted 3D bounding boxes are projected to the 2D image plane. A learnable refinement module is then used to adjust these projected boxes, ensuring tighter alignment with ground-truth 2D boxes and enabling geometry-aware supervision.
Datasets
REXO is evaluated on two publicly available indoor radar datasets:
- MMVR (Multi-View Radar Dataset): This dataset features multi-view radar heatmaps collected from over 25 human subjects across 6 rooms over 9 days. It includes protocols for open foreground and cluttered spaces, with data splits for random and cross-session evaluation.
- HIBER: This dataset includes multi-view radar heatmaps from 10 human subjects in a single room, captured from different angles. The "WALK" data split, consisting of 73.5K frames, was used for evaluation.
Both datasets provide annotations such as 3D bounding boxes in radar coordinates and 2D bounding boxes in the image plane.
Key Insights and Ideas
- 3D Diffusion is Key: Lifting the diffusion process to 3D bounding boxes in radar space is crucial for effective cross-view association and robust denoising in this domain.
- Explicit Guidance Works: Explicitly guiding the cross-view feature association using projected 3D boxes significantly improves performance compared to implicit methods.
- Prior Knowledge Matters: Incorporating simple prior knowledge, like the ground-level constraint, can effectively reduce model complexity and improve detection accuracy and convergence speed.
- Refinement is Necessary: Even with accurate 3D radar predictions, a refinement module for projected 2D boxes is essential to achieve high performance on the image plane due to projection inaccuracies.
Key Results
REXO demonstrates significant performance improvements on both MMVR and HIBER datasets:
- MMVR: REXO surpasses state-of-the-art methods by margins of +4.22 AP on the HIBER dataset and +11.02 AP on the MMVR dataset. Specifically, on the challenging "P2S2" split (unseen environments), REXO boosted AP from 12.45 to 23.47 compared to the best baseline (RETR).
- HIBER: REXO achieved an AP of 25.33 on the "WALK" split, outperforming RETR's AP of 22.09 and demonstrating strong performance across different IoU thresholds.
- Robustness: REXO shows better robustness to varying numbers of bounding boxes during inference compared to RETR.
- Convergence: The ground-level constraint leads to faster convergence during training.
Limitations and Future Work
- Ground-Level Constraint Accuracy: The authors acknowledge that the ground-level constraint might not always be accurate when subjects jump or stand on obstacles.
- Performance on "Seen" Frames: On simpler "seen" frame configurations (like P1S1), REXO's performance was slightly lower than RETR, particularly on AP50, possibly due to the nature of the diffusion process and its probabilistic sampling.
- Failure Cases: Visualizations of failure cases show that REXO can still produce inaccurate positions or false negatives, especially in scenarios with coarse resolution (distant objects) or complex overlapping reflections.
- Future Work: The paper suggests that further exploration of the trade-off between 2D and 3D supervision strengths and dynamic denoising steps could lead to further improvements. Investigating more sophisticated prior knowledge or adaptive constraints could also be beneficial.
Summary updated