3D Object Detection under Urban Road Traffic Scenarios Based on Dual-Layer Voxel Features Fusion Augmentation

Haobin Jiang, Junhao Ren, Aoxue Li2024 · DOI

TL;DR

This paper introduces DL-VFFA, a novel dual-layer voxel feature fusion network for 3D object detection in urban traffic. By enhancing voxel feature representation and intelligently fusing LiDAR and camera data, DL-VFFA significantly improves detection accuracy, especially for occluded and distant objects, outperforming existing methods on the KITTI dataset.

What Problem Does This Paper Address?

Autonomous vehicles rely heavily on accurate 3D object detection to perceive their surroundings for safe navigation. While LiDAR provides precise depth information and cameras offer rich semantic context (color, texture), effectively fusing these heterogeneous data sources remains a significant challenge. Existing methods often struggle with local occlusions and limited fields of view, leading to misrecognitions. Furthermore, voxelization, a common technique for processing LiDAR point clouds, can lead to the loss of fine-grained local features and disconnect objects. This paper aims to tackle these issues by proposing a more robust fusion strategy that preserves and enhances local object details.

Methods

The proposed DL-VFFA network introduces several key innovations:

  • Dual-Layer Voxel Feature Fusion:
    • Mahalanobis Distance for Voxel Association: To mitigate the loss of local features during voxelization, the paper uses the Mahalanobis distance to associate boundary points within a voxel with points in neighboring voxels. This helps create voxel feature mappings that are better aligned with local object information.
  • Neighborhood Voxel (NV) and Global Voxel (GV) Modules: These modules are integrated into the voxelization process. NV focuses on local voxel details, while GV consolidates global voxel insights. An improved channel attention mechanism aggregates features from both NV and GV, which are then spliced with the original voxelized features.
  • Improved Attention Gaussian Deviation Matrix (GDM): This module is used to compute relative position encodings for voxel features. It dynamically adjusts the Gaussian deviation matrix based on input data characteristics, enhancing the network's ability to extract key point information and adapt to different voxel sequences.
  • Learnable Weight Parameters (LWP): During the fusion of point cloud and image features, learnable weights are introduced to decouple fine-grained regions. This allows for a two-layer feature fusion from voxel-to-voxel and point cloud-to-image, enhancing feature information at key points.
  • LiDAR and Image Feature Fusion Module: This module employs a dual convolution fusion operator. It first identifies fine-grained regions in object category labels, then divides them into local regions for multi-region fine-grained fusion. An improved channel attention mechanism adaptively learns weights for different feature channels, allowing the network to focus on the most relevant information from both LiDAR and camera data.

Datasets

The primary dataset used for training and evaluation in this paper is the KITTI dataset. The authors used a 0.7:0.3 train-validation split. Evaluation was conducted across different difficulty levels (easy, moderate, hard) and for three object classes: cars, pedestrians, and cyclists.

Key Insights and Ideas

  • Voxelization isn't perfect: The paper highlights that standard voxelization can lose crucial local details, which is a significant problem for detecting occluded or distant objects. Their Mahalanobis distance approach directly addresses this by explicitly linking neighboring voxel information.
  • Attention is key for multimodal fusion: The use of improved channel attention mechanisms, both within voxel processing and for fusing LiDAR and image data, is central to DL-VFFA's success. This allows the network to intelligently weigh and combine information from different sources.
  • Dual-layer fusion offers richer representations: By processing features at both local voxel and global voxel levels, and then fusing them with image features in a layered manner, the network can build more comprehensive and accurate object representations.
  • Adaptive learning for robustness: The dynamic adjustment of parameters (like the Gaussian deviation matrix and channel attention weights) allows the model to adapt to varying scene conditions, leading to better performance across different scenarios.

Key Results

DL-VFFA demonstrates significant improvements over baseline methods, particularly MVX-Net, on the KITTI dataset:

  • Overall Performance: DL-VFFA achieves better results in detecting objects in distant scenes and in scenarios with occlusions.
  • Distance-Based Performance: Compared to MVX-Net, DL-VFFA shows average precision increases of 4.9% to 7.8% at distances of 20-50m.
  • Occlusion-Based Performance: For moderate and severe occlusion levels, DL-VFFA achieves average precision increases of 5.5% and 6.4% (3D-AP), and 4.5% and 7.5% (BEV-AP), respectively, compared to MVX-Net.
  • Voxel Fusion Module: The proposed voxel fusion module achieves a mAP of 79.48% (IOU=0.5) and 72.25% (IOU=0.7), outperforming MVX-Net by 3.21% and 1.54%, respectively, while maintaining a competitive inference speed of 11.0 FPS.
  • Ablation Studies: Adding the N-GV, GDM, and LWP modules individually to the baseline "Second" network resulted in mAP improvements of 1.37%, 1.74%, and 1.46%, respectively. Combining all modules yielded a significant 3.93% mAP improvement.

Limitations and Future Work

  • Limitations:
    • The paper notes that in one specific scene (Figure 9a), DL-VFFA still missed some vehicles, suggesting potential issues in calculating image and point cloud feature weights, which will be addressed in future work.
  • While the model achieves good real-time performance, further optimization might be possible.
  • Future Work:
    • The authors suggest further improving the all-weather perception capability of the object detection system, particularly focusing on enhancing target detection accuracy under low-light conditions.

Summary updated

BibTeX
@misc{jiang2024d,
  author = {Haobin Jiang and Junhao Ren and Aoxue Li},
  title = {{3D Object Detection under Urban Road Traffic Scenarios Based on Dual-Layer Voxel Features Fusion Augmentation}},
  year = {2024},
  doi = {10.3390/s24113267},
}
Figure