Abstract
Multi-view 3D object detection is a critical task for autonomous driving, yet current sparse query-based paradigms suffer from redundant global attention and complex coordinate mapping. We present Divided View Position Embedding (DVPE), a framework that partitions the 3D world into local virtual spaces. By decoupling spatial learning from camera poses and implementing visibility-aware cross-attention, DVPE reduces interference and achieves state-of-the-art performance on the nuScenes benchmark, reaching 57.2% mAP and 64.5% NDS.
Introduction
In autonomous driving, reconstructing a 3D scene from a set of 2D surround-view cameras is a fundamental challenge. Modern architectures often employ sparse query-based methods, where 3D object queries interact with multi-view image features. However, these methods face two primary bottlenecks. First, redundant attention occurs when global attention mechanisms force every 3D query to look at all pixels across all cameras, introducing significant noise from irrelevant views. Second, coordinate complexity arises because the model must learn to map 3D world coordinates back to 2D image planes across varying camera poses, hindering convergence.
Methodology
The DVPE framework introduces a "divide and conquer" strategy to address these challenges through three core components.
Divided View Position Embedding (DVPE)
Instead of a monolithic global coordinate system, DVPE partitions the 3D space into multiple frustum-shaped local virtual spaces aligned with each camera's field of view. By transforming 3D reference points into these localized coordinates, the model decouples position embeddings from specific extrinsic camera parameters. This geometric simplification allows the transformer to learn spatial relationships more efficiently.
Visibility Cross-Attention
Leveraging the divided virtual spaces, the model implements a visibility mechanism. Queries are restricted to performing cross-attention only with image features that fall within their corresponding virtual space. This targeted focus mimics human perception and drastically reduces computational interference from non-overlapping views.
Enhanced Temporal and Supervision Strategies
To improve object tracking, DVPE incorporates 2D historical RoI features from a 2D detector, providing high-level semantic anchors for temporal fusion. Additionally, a one-to-many assignment strategy is used during training, providing denser supervision signals to the decoder and stabilizing the learning process.
Results and Discussion
DVPE was evaluated on the nuScenes dataset, the industry standard for multi-view 3D detection. The model achieved state-of-the-art results with 57.2% mAP and 64.5% NDS on the test set. Ablation studies highlight the efficacy of the divided view approach, which alone contributed a 2.4% mAP improvement over the StreamPETR baseline.
The primary insight of this work is that spatial localization—restricting the model's field of regard—paradoxically enhances its intelligence by filtering out environmental noise. By simplifying the underlying trigonometry through local virtual spaces, DVPE allows the network to focus on the semantic task of object detection rather than geometric reconstruction.
Conclusion
DVPE demonstrates that partitioning 3D space into camera-specific virtual volumes significantly improves the accuracy and efficiency of multi-view 3D object detection. By addressing the noise inherent in global attention and the complexity of camera-to-world mapping, DVPE sets a new standard for sparse query-based perception in autonomous systems.
Summary updated