PSANet: Pyramid Splitting and Aggregation Network for 3D Object Detection in Point Cloud

Fangyu Li, Weizheng Jin, Cien Fan, Lian Zou, Qingsheng Chen, Xiaopeng Li, Hao Jiang, Yifeng Liu2021 · DOI

TL;DR

This paper introduces PSANet, a novel backbone network for 3D object detection in LiDAR point clouds. PSANet addresses the limitations of existing methods by effectively fusing multi-scale Bird's Eye View (BEV) features through a pyramid splitting and aggregation module, leading to improved detection accuracy and robustness.

What Problem Does This Paper Address?

3D object detection in LiDAR point clouds is crucial for autonomous driving, robotics, and augmented reality. While one-stage detectors offer speed advantages, they often struggle with insufficient utilization of Bird's Eye View (BEV) information, leading to performance issues, especially for distant or occluded objects. Existing methods often simplify the Region Proposal Network (RPN) or struggle with aligning sparse point cloud data with image information. This paper aims to improve the accuracy and robustness of 3D object detection by proposing a new backbone network that effectively fuses multi-scale BEV features.

Methods

The core contribution is the PSANet architecture, which features a novel backbone network called PFH-PSA. This backbone is divided into two branches:

  • Coarse Branch (Pyramidal Feature Hierarchy - PFH): This branch generates multi-scale BEV feature maps, capturing both low-level texture information and high-level semantic information. This helps in reducing false detections caused by background noise.
  • Fine Branch (Pyramid Splitting and Aggregation - PSA): This module performs cross-layer fusion of the multi-scale feature maps from the coarse branch, enhancing the expressive ability of the final features.

The overall PSANet detector follows a standard pipeline:

  1. Data Preprocessing: Point clouds are voxelized and processed.
  2. Voxel-wise Feature Extractor: A simple mean-based extractor is used.
  3. 3D Sparse Convolutional Middle Extractor: This part compresses the 3D sparse data into a dense 2D BEV pseudo-image.
  4. Reshaping to BEV: The 3D sparse tensor is converted into a 2D BEV representation.
  5. Cross-Layer Fusion (PFH-PSA): The proposed backbone network fuses multi-scale BEV features.

The detection head uses a standard approach with anchor-based classification, bounding box regression, and direction classification, employing Focal loss for classification and Smooth-L1 loss for regression.

Datasets

The paper uses the KITTI-3D benchmark for training and evaluation. The KITTI dataset is a widely used benchmark for autonomous driving, containing real-world data with varying scenes, object densities, and occlusion levels. The authors divide the KITTI dataset into training and validation sets and evaluate performance across three difficulty levels: easy, moderate, and hard.

Key Insights and Ideas

  • Effective BEV Feature Fusion is Key: The paper strongly emphasizes that simply extracting multi-scale BEV features is not enough. The real improvement comes from intelligently fusing these features across different layers, as demonstrated by the PSA module.
  • Synergistic Effect of Coarse and Fine Branches: The coarse branch provides essential low-level and high-level features, while the fine branch enhances their expressiveness through cross-layer fusion. Both branches are crucial and work synergistically for optimal 3D and BEV detection.
  • Robustness to Occlusion and Truncation: The proposed method shows improved robustness in detecting heavily occluded or truncated objects, outperforming other state-of-the-art voxel-based methods in challenging scenarios.
  • Balancing Speed and Accuracy: PSANet achieves competitive detection performance while maintaining a reasonable inference speed (11 FPS on a GTX 1080Ti GPU), making it a viable option for real-time applications.

Key Results

  • PSANet achieves the best performance across different difficulty levels in 3D object detection tasks on the KITTI validation set, outperforming even some two-stage methods.
  • In BEV object detection, PSANet achieves sub-optimal results, comparable to the most advanced methods.
  • The proposed method achieves an average precision of 89.02% (Easy), 78.70% (Moderate), and 77.57% (Hard) for 3D detection and 90.20% (Easy), 87.88% (Moderate), and 86.20% (Hard) for BEV detection.
  • The inference speed reaches 11 FPS on a single GTX 1080Ti GPU.

Limitations and Future Work

  • Small Target Detection: The paper acknowledges limitations in detecting small targets, which are often overlooked due to sparse point clouds and complex contours. Future work will explore fusing images with point clouds to improve small target detection.
  • Inference Speed for High-Speed Vehicles: While 11 FPS is competitive, it's still insufficient for high-speed vehicle detection and tracking. The authors plan to use TensorRT for acceleration and explore better GPUs.
  • Generalization: Most state-of-the-art methods are evaluated only on KITTI. Future research will focus on developing a unified framework that can combine multiple datasets (e.g., Waymo, nuScenes) to improve generalization.
  • Multi-Sensor Fusion: The authors suggest exploring more effective multi-sensor fusion methods, potentially by generating high-quality pseudo-point clouds from color images to complete distant targets.
  • Attention Mechanisms: Incorporating point cloud attention and voxel attention is proposed to further refine detection by avoiding background or noise points.

Summary updated

BibTeX
@misc{li2021psanet,
  author = {Fangyu Li and Weizheng Jin and Cien Fan and Lian Zou and Qingsheng Chen and Xiaopeng Li and Hao Jiang and Yifeng Liu},
  title = {{PSANet: Pyramid Splitting and Aggregation Network for 3D Object Detection in Point Cloud}},
  year = {2021},
  doi = {10.3390/s21010136},
}
Figure