Rethinking Dimensionality Reduction in Grid-based 3D Object Detection

Dihe Huang, Ying Chen, Yikang Ding, Jinli Liao, Jianlin Liu, Kai Wu, Qiang Nie, Yong Liu, Chengjie WangarXiv 2022 · DOI · OpenReview

TL;DR

This paper introduces MDRNet, a novel backbone for 3D object detection that addresses the critical loss of 3D spatial information in current grid-based methods. By employing Spatial-aware Dimensionality Reduction (SDR) and Multi-level Spatial Residuals (MSR), MDRNet effectively retains and fuses 3D geometric cues, leading to state-of-the-art performance on challenging benchmarks without sacrificing efficiency.

What Problem Does This Paper Address?

Detecting 3D objects from point cloud data is crucial for applications like autonomous driving. Many successful methods convert 3D point clouds into a Bird's Eye View (BEV) representation for efficient processing using 2D convolutional networks. However, this conversion typically involves collapsing the 3D data along the height (Z) dimension, which leads to a significant loss of valuable 3D spatial and geometric information. This information loss is particularly detrimental for objects with complex shapes and varying sizes, such as motorcycles and bicycles, hindering accurate detection and localization. Existing methods often use fixed pooling or convolution operations that don't adapt to the diverse geometric structures of objects, further exacerbating this problem.

Methods

The paper proposes MDRNet, a new backbone architecture designed to mitigate the information loss during the 3D-to-BEV dimensionality reduction. MDRNet features a dual-branch structure: a voxel branch for processing 3D features and a BEV branch for processing 2D features. The core innovations are two new modules:

  • Spatial-aware Dimensionality Reduction (SDR): This module dynamically focuses on important object parts during the voxel-to-BEV transformation. It learns a spatial distribution of voxel features along the height dimension, allowing the network to adaptively aggregate features based on their geometric and semantic relevance. This contrasts with static methods that use fixed pooling or convolution weights.
  • Multi-level Spatial Residuals (MSR): This module fuses multi-level spatial information by integrating voxel features with BEV features at each stage of the network. This cross-dimensional connection helps enrich the BEV features with retained 3D geometric information, improving perception across different resolutions.

MDRNet can be easily integrated into existing grid-based 3D object detection frameworks by replacing their backbone.

Datasets

The proposed method is evaluated on two widely used autonomous driving datasets:

  • nuScenes: A large-scale, multimodal dataset with 3D bounding box annotations for 10 object categories. The authors use this dataset for extensive performance evaluation and ablation studies.
  • KITTI: A benchmark dataset commonly used for 3D object detection, featuring 3 categories (car, pedestrian, cyclist) with varying difficulty levels. This dataset is used for evaluating improvements on existing detectors.

Key Insights and Ideas

  • Dimensionality Reduction is a Bottleneck: The paper highlights that the standard practice of collapsing 3D point cloud data to BEV is a significant source of information loss, especially for complex objects.
  • Adaptivity is Key: Static dimensionality reduction methods fail to capture the diverse geometric properties of different objects. SDR's dynamic, spatial-aware approach is crucial for adaptive feature aggregation.
  • Multi-level Fusion Matters: Retaining 3D geometric information not just at the initial or final stages, but across multiple resolutions (via MSR), significantly boosts detection performance.
  • Efficiency Preserved: The proposed MDRNet achieves substantial performance gains without increasing computational runtime, making it practical for real-world applications.

Key Results

  • nuScenes Performance: MDRNet, when integrated with CenterPoint [13], achieves a significant improvement, reaching 65.2% mAP and 70.5% NDS on the nuScenes test set. This represents a substantial boost over the baseline.
  • Complex Object Improvement: The method shows particularly strong gains for motorcycle and bicycle detection (73.1% and 45.2% AP, respectively), demonstrating its effectiveness in handling objects with complex geometries.
  • KITTI Improvement: MDRNet also shows appreciable improvements on the KITTI dataset, particularly for the pedestrian category, boosting AP3D from 54.49% to 60.06%.
  • Runtime Efficiency: The inference speed of MDRNet is comparable to the original CenterPoint, indicating no significant computational overhead.
  • Ablation Studies: Experiments confirm that SDR-Softmax is the most effective normalization for SDR, and that incorporating MSR across more stages leads to better performance.

Limitations and Future Work

  • SDR Stages: While MSR is beneficial across multiple stages, the paper notes that SDR is primarily used in the first stage. The rationale is that subsequent stages have downsampled feature spaces where sparsity adaptation is less critical. Future work could explore the benefits of applying SDR more broadly.
  • Multi-modal Fusion: The paper mentions a multi-modal variant that uses a simple fusion mechanism. While effective, exploring more sophisticated fusion strategies for multi-modal data could potentially yield further improvements.
  • Generalizability: While MDRNet is presented as a universal backbone, further testing on a wider range of grid-based detectors and diverse datasets would solidify its general applicability.

Summary updated

Abstract

Bird's eye view (BEV) is widely adopted by most of the current point cloud detectors due to the applicability of well-explored 2D detection techniques. However, existing methods obtain BEV features by simply collapsing voxel or point features along the height dimension, which causes the heavy loss of 3D spatial information. To alleviate the information loss, we propose a novel point cloud detection network based on a Multi-level feature dimensionality reduction strategy, called MDRNet. In MDRNet, the Spatial-aware Dimensionality Reduction (SDR) is designed to dynamically focus on the valuable parts of the object during voxel-to-BEV feature transformation. Furthermore, the Multi-level Spatial Residuals (MSR) is proposed to fuse the multi-level spatial information in the BEV feature maps. Extensive experiments on nuScenes show that the proposed method outperforms the state-of-the-art methods. The code will be available upon publication.
BibTeX
@misc{huang2022rethinking,
  author = {Dihe Huang and Ying Chen and Yikang Ding and Jinli Liao and Jianlin Liu and Kai Wu and Qiang Nie and Yong Liu and Chengjie Wang},
  title = {{Rethinking Dimensionality Reduction in Grid-based 3D Object Detection}},
  year = {2022},
  eprint = {2209.09464},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2209.09464},
}
Figure