Context-Aware Dynamic Feature Extraction for 3D Object Detection in Point Clouds

Yonglin Tian, Lichao Huang, Hui Yu, Xiangbin Wu, Xuesong Li, Kunfeng Wang, Zilei Wang, Fei-Yue WangIEEE Trans. ITS 2022 · OpenReview

TL;DR

This paper introduces CADNet, a novel 3D object detection system for point clouds that tackles the challenge of varying point density. By incorporating "context features" that capture a wider spatial area and employing "decomposable dynamic convolutions" that adapt their filters based on local input, CADNet achieves state-of-the-art performance on the KITTI dataset while maintaining a fast inference speed.

What Problem Does This Paper Address?

Detecting objects in 3D point clouds, particularly from LiDAR sensors, is crucial for applications like autonomous driving. A significant challenge is the varying density of points that make up objects. As an object moves further away from the sensor, the number of points representing it decreases, leading to sparser data. Traditional convolutional neural networks (CNNs) struggle with this because they use fixed filters that are applied uniformly across all spatial locations. This means a filter designed for dense point clouds might not work well for sparse ones, and vice-versa, hindering accurate feature extraction and object detection.

Methods

CADNet introduces two main innovations to address the varying density problem:

  • Context Feature Extraction: The authors extend the standard voxelization approach by not only processing individual "pillars" (voxelized columns) but also extracting "context features." These context features are derived from points within a larger spatial region around each pillar (3x the width and length). This wider receptive field allows the network to capture more information about the local environment and the object's density variations. These context features are processed in parallel with pillar features and are used to generate a "guidance map" that helps focus attention on important regions.
  • Decomposable Dynamic Convolution (DD-Conv): To further adapt to local feature variations, the paper proposes DD-Conv. Unlike traditional convolutions with fixed kernels, DD-Conv dynamically generates its filter weights based on the input. It achieves this by decomposing the dynamic filter generation into two parts:
  • Shared Weights: These are static kernels that capture common patterns across all inputs, similar to traditional convolutions.
  • Dynamic Weights: These are generated by a coefficient generator that learns position-dependent coefficients for multiple static kernels. This allows the filters to adapt their behavior based on the specific local input features and their semantic context. This approach significantly reduces the number of parameters compared to other dynamic filter methods.

These components are integrated into a dual-path region proposal network (RPN) and a detection head to produce the final 3D bounding boxes.

Datasets

The paper primarily uses the KITTI dataset for training and evaluation. They report results on both the validation and test sets for 3D and Bird's Eye View (BEV) detection tasks, specifically for the "car" and "cyclist" classes.

Key Insights and Ideas

  • Density is a Key Challenge: The paper clearly articulates that varying point density is a fundamental obstacle in 3D point cloud object detection, impacting the effectiveness of standard CNNs.
  • Context is Crucial: Expanding the receptive field beyond individual pillars to include surrounding context is vital for understanding local density variations and improving feature representation.
  • Dynamic Filters for Adaptability: The core idea of dynamically adjusting convolutional filters based on input is powerful. The DD-Conv's decomposable approach offers a more parameter-efficient way to achieve this adaptability.
  • Shared + Dynamic = Robustness: Combining static (shared) filters with dynamic filters proves beneficial. The shared filters capture general object characteristics, while dynamic filters handle specific variations, leading to more robust detection.
  • Guidance Maps Improve Focus: Using context features to generate guidance maps helps the network prioritize relevant regions, potentially improving efficiency and accuracy.

Key Results

  • State-of-the-Art Performance: CADNet achieves competitive performance on the KITTI dataset, outperforming established methods like SECOND and PointPillars by a significant margin on 3D detection tasks.
  • High FPS: The proposed method achieves a fast inference speed of 30 FPS, making it suitable for real-time applications.
  • Parameter Efficiency: The DD-Conv significantly reduces the number of parameters compared to other dynamic filter methods, making it more practical for deployment. For example, they report a memory usage reduction of over 10,000 times compared to DFN.
  • Ablation Studies Confirm Contributions: Ablation studies demonstrate that both the context features and the DD-Conv contribute positively to performance, with their combination yielding the best results.

Limitations and Future Work

  • Coefficient Generator Simplicity: The authors note that when using a large number of static kernels (M), the performance can decrease. They attribute this to the simplicity of their coefficient learning network (Gdc), suggesting that learning the relationship between too many tensors can be challenging. This suggests potential for improvement in the coefficient generation module.
  • False Detections: Visualizations show that false detections can occur due to confusing background elements (like walls) or objects with sparse point clouds at far distances.
  • Labeling Issues: The paper mentions that missing labels in the KITTI dataset can also contribute to false cases.

Future work could focus on improving the coefficient generator for more complex scenarios and exploring ways to better handle extremely sparse point clouds or ambiguous background clutter.

Summary updated

Abstract

Varying density of point clouds increases the difficulty of 3D detection. In this paper, we present a context-aware dynamic network (CADNet) to capture the variance of density by considering both point context and semantic context. Point-level contexts are generated from original point clouds to enlarge the effective receptive filed. They are extracted around the voxelized pillars based on our extended voxelization method and processed with the context encoder in parallel with the pillar features. With a large perception range, we are able to capture the variance of features for potential objects and generate attentive spatial guidance to help adjust the strengths for different regions. In the region proposal network, considering the limited representation ability of traditional convolution where same kernels are shared among different samples and positions, we propose a decomposable dynamic convolutional layer to adapt to the variance of input features by learning from the local semantic context. It adaptively generates the position-dependent coefficients for multiple fixed kernels and combines them to convolve with local features. Based on our dynamic convolution, we design a dual-path convolution block to further improve the representation ability. We conduct experiments on KITTI dataset and the proposed CADNet has achieved superior performance of 3D detection outperforming SECOND and PointPillars by a large margin at the speed of 30 FPS.
BibTeX
@article{tian2022context,
  author = {Yonglin Tian and Lichao Huang and Hui Yu and Xiangbin Wu and Xuesong Li and Kunfeng Wang and Zilei Wang and Fei-Yue Wang},
  title = {{Context-Aware Dynamic Feature Extraction for 3D Object Detection in Point Clouds}},
  journal = {IEEE Trans. Intell. Transp. Syst.},
  year = {2022},
  url = {https://openreview.net/forum?id=DuVbXPDMOV},
}
Figure