Abstract
Multi-view 3D object detection (MV3D-Det) in Bird-Eye-View (BEV) is a cost-effective alternative to LiDAR-based systems but suffers from significant performance degradation when deployed across different camera configurations. This "domain gap" is primarily driven by depth-shift errors caused by variations in camera intrinsics and extrinsics. We present DG-BEV, a framework that addresses these challenges through intrinsics-decoupled depth prediction, dynamic perspective augmentation, and adversarial domain-invariant feature learning. Experimental results across nuScenes, Waymo, and Lyft datasets demonstrate that DG-BEV achieves up to a 7x increase in detection accuracy on unseen domains without sacrificing source domain performance.
1. Introduction
Multi-view 3D object detection (MV3D-Det) has emerged as a critical component for autonomous driving systems seeking to minimize hardware costs. Despite its potential, MV3D-Det models are notoriously sensitive to variations in camera parameters, including focal lengths, mounting heights, and angles. When a model trained on a specific fleet is deployed on a vehicle with a different sensor setup, performance typically collapses. This degradation is largely attributed to a shift in the BEV feature distribution, specifically a "depth-shift" phenomenon where objects are incorrectly localized along the depth axis due to the model's reliance on specific training camera geometries.
2. Methodology
The DG-BEV framework addresses the domain gap through three integrated strategies designed to make the detection pipeline domain-agnostic.
2.1 Intrinsics-Decoupled Depth Prediction (IDD)
Standard depth estimation modules often learn to associate specific image scales with metric depths, effectively "hard-coding" the training camera's focal length into the neural network weights. DG-BEV instead predicts a scale-invariant depth and uses the camera's known focal length as a mathematical bridge to recover metric depth. This prevents the model from miscalculating distance when the focal length changes.
2.2 Dynamic Perspective Augmentation (DPA)
To ensure robustness against varying extrinsic parameters, such as camera mounting heights and angles, the framework employs homography-based warping. This simulates diverse camera orientations (pitch, yaw, and roll) during training, forcing the model to learn the underlying geometry of the scene rather than memorizing a fixed viewpoint.
2.3 Domain-Invariant Feature Learning (DIFL)
DG-BEV creates "pseudo-domains" by artificially scaling images to simulate different focal lengths. An adversarial setup using a Gradient Reverse Layer (GRL) is then employed to "punish" the model if it can distinguish between these domains. This encourages the backbone to extract features that are useful for detection but invariant to camera-specific quirks. The researchers found that using an Ordinal Loss for these pseudo-domains performed best, as focal length is a continuous physical property.
3. Experimental Results
The framework was evaluated using nuScenes as the primary source domain, with the Waymo Open Dataset and Lyft L5 Dataset as target domains to test cross-dataset generalization.
3.1 Cross-Dataset Generalization
The results demonstrate a dramatic improvement in zero-shot generalization. For the nuScenes to Waymo transition, the baseline model achieved a Mean Average Precision (mAP) of only 0.040. DG-BEV improved this to 0.297—a massive gain that makes the model viable in the new domain. Similarly, for Waymo to nuScenes, mAP increased from 0.032 to 0.303.
3.2 Source Domain Stability
Crucially, the improvements in generalization did not come at the cost of performance on the original training data. The model maintained its original accuracy levels on the source domain, indicating that the domain-agnostic features are equally effective for the training distribution.
4. Conclusion
DG-BEV successfully mitigates the geometric domain gap in MV3D-Det by decoupling depth estimation and enforcing feature invariance. While this work focuses on geometric projections, future research should address the semantic gap—differences in environmental conditions, lighting, and regional object appearances—to develop a truly universal 3D detection system.
Summary updated