Revisiting Cross-Domain Problem for LiDAR-Based 3D Object Detection

Ruixiao Zhang, Juheon Lee, Xiaohao Cai, Adam Prugel-BennettCommunications 2025 · DOI · OpenAlex

Introduction

3D object detection is a cornerstone of autonomous driving, yet models often experience significant performance degradation when deployed in environments different from their training data—a challenge known as the cross-domain problem. This paper investigates the root causes of this failure, revealing that current models tend to overfit to the specific object dimensions of the source dataset rather than learning generalizable geometric features.

The Cross-Domain Challenge

For autonomous systems to scale, they must operate reliably across various cities, weather conditions, and sensor configurations. However, a model trained on one dataset (source domain) typically fails when tested on another (target domain). While domain adaptation techniques exist, the authors suspect they often act as a "band-aid," masking structural flaws in how models perceive spatial data. They set out to analyze whether adding camera data helps or hurts and whether current evaluation metrics are capturing the root of the problem.

Methodology and New Metrics

The authors evaluate several 3D detection architectures, including point-based (PV-RCNN), voxel-based (SECOND-IoU), and transformer-based (TransFusion) models. To better diagnose spatial failures, they propose two novel metrics beyond standard 3D or Bird's-Eye View (BEV) Average Precision (AP):

  1. Side-view AP: Evaluates length and height accuracy by projecting boxes onto the X-Z plane.
  2. Front-view AP: Evaluates width and height accuracy by projecting boxes onto the Y-Z plane.

Datasets

The study leverages three major benchmarks, standardizing coordinate systems and voxel sizes to allow for cross-dataset evaluation:

  • KITTI: Features 64-beam LiDAR with dense point clouds, focused on front-view annotations.
  • nuScenes: Utilizes 32-beam LiDAR, resulting in sparser point clouds.
  • Waymo: A large-scale dataset providing high-density point clouds.
Comparison of LiDAR and camera data across the KITTI, Waymo, and nuScenes datasets.

Key Insights

The Fragility of Multi-Modal Models

Contrary to intuition, integrating camera data with LiDAR often decreases robustness in cross-domain scenarios. Variations in camera calibration, image resolution, and pixel distributions make multi-modal models more prone to failure than their LiDAR-only counterparts when encountering new domains. For example, a multi-modal TransFusion model collapsed to 0.5% AP on a nuScenes-to-KITTI task, while its LiDAR-only version maintained 17.4%.

The Illusion of Domain Adaptation

Evaluation of the ST3D domain adaptation method suggests it does not necessarily improve fundamental generalization. Instead, it often causes the model to "forget" the source domain and overfit to the target domain, leading to unstable performance depending on the specific subset of target data used.

The Width vs. Length Paradox

A critical finding is that models struggle significantly more with predicting object width across domains than length. Using the proposed front-view AP, the authors discovered that models often memorize the average car width of the training set. Because Waymo cars are roughly 30% wider than KITTI cars, a model trained on Waymo will incorrectly apply those dimensions to KITTI vehicles. In one test, a model achieved a Length AP of 73.9% but a Width AP of only 12.1%.

Conclusion

The authors conclude that the community must move beyond simple 3D/BEV AP metrics, which can hide dimensional overfitting. Future research should focus on developing architectures that perform true geometric reasoning from point clouds rather than memorizing statistical dimensions, ensuring safety and reliability across diverse real-world environments.

Summary updated

BibTeX
@inproceedings{zhang2025revisiting,
  author = {Ruixiao Zhang and Juheon Lee and Xiaohao Cai and Adam Prugel-Bennett},
  title = {{Revisiting Cross-Domain Problem for LiDAR-Based 3D Object Detection}},
  booktitle = {Communications},
  year = {2025},
  doi = {10.1007/978-981-96-7036-9\_6},
}
Figure