PCT: Large-Scale 3d Point Cloud Representations Via Graph Inception Networks with Applications to Autonomous Driving

Siheng Chen, Sufeng Niu, Tian Lan, Baoan Liu

TL;DR

This paper introduces the Point Cloud Neural Transform (PCT), a novel system for efficiently representing large-scale 3D point clouds by combining voxelization with graph neural networks. PCT significantly improves upon existing methods for tasks like autonomous driving by avoiding discretization errors and capturing complex spatial distributions.

What Problem Does This Paper Address?

Representing large-scale 3D point clouds, especially those from sensors like LiDAR used in autonomous driving, is a significant challenge. Traditional methods like voxelization can introduce errors and struggle with the vast variations in real-world scenarios. Other approaches, while learning-based, can be computationally expensive or fail to capture the irregular, sparse nature of point cloud data. The goal is to find a compact, yet informative, representation that facilitates downstream tasks like compression, localization, and object detection.

Methods

The proposed Point Cloud Neural Transform (PCT) employs a two-stage approach:

  1. Voxelization: The large 3D space is first divided into smaller, non-overlapping voxels, similar to how images are processed in blocks. This breaks down the problem into more manageable, local regions.
  2. Voxel-Level Encoding: Within each voxel, novel Graph Inception Networks (GIN) are used to learn representations. GINs treat 3D points as nodes in a graph, connecting neighboring points to capture their spatial relationships. This approach is designed to be:
  3. Translation-invariant: The representation doesn't change if the points are shifted.
  4. Permutation-invariant: The order of points within a voxel doesn't affect the output.
  5. Scale and Rotation Equivalent: Normalization networks are used to ensure that variations in scale and rotation are handled explicitly, allowing the GINs to focus on local shape.

The GINs themselves use a series of "inception" layers, which apply graph convolutions with multiple neighborhood sizes (K-nearest neighbors) to capture features at different scales within the voxel. The output from these convolutions is then aggregated (mean aggregation) to produce a single feature vector for the voxel. The entire PCT system is analogous to the blocked discrete cosine transform used for 2D images.

Datasets

  • KITTI: The paper uses the KITTI dataset, a standard benchmark for autonomous driving research. Specifically, they utilize real-time LiDAR sweeps collected by a Velodyne HDL-64E scanner.
  • The dataset was split into 400 sweeps for training and 100 for testing, with care taken to avoid data snooping by selecting sweeps from separate driving logs.

Key Insights and Ideas

  • Bridging Voxelization and Learning: The core innovation is combining the structured partitioning of voxelization with the rich feature learning capabilities of graph neural networks. This hybrid approach aims to get the best of both worlds: avoiding the discretization issues of pure voxelization and the complexity of end-to-end learning on raw, large-scale point clouds.
  • Graph Inception Networks for Local Structure: The GINs are crucial for capturing the intricate, non-uniform distributions of points within a voxel. By using multiple K-nearest neighbor graph sizes, they effectively act like multi-scale filters in the graph domain.
  • Analogy to Image Transforms: The paper draws a strong parallel between PCT and established image processing techniques like the blocked discrete cosine transform. This analogy helps to intuitively understand PCT's role in transforming raw point cloud data into a more compact and usable representation.
  • Importance of Mean Aggregation: The authors found that mean aggregation of voxel features (as opposed to max aggregation) preserves richer information for reconstruction, leading to better performance.

Key Results

  • Superior Reconstruction Performance: The proposed PCT, particularly when using GINs, significantly outperforms competing methods (including uniform resampling, K-means based resampling, PointNet, and DGCNN) in terms of reconstruction quality across various metrics like Earth Mover's Distance (EMD), Chamfer Distance (CD), Mean Squared Error (MSE), Mean, and Variance.
  • Ablation Study Validation: An ablation study confirmed that each component of PCT (Graph Inception, Mean-Aggregation, Normalization) contributes positively to the overall performance. Fully-connected layers were found to be a more effective decoder than FoldingNet.
  • Effective Compression: The PCT achieves good compression ratios (e.g., 3.19% code length in Table 1) while maintaining high reconstruction fidelity, as demonstrated by the mean square error plots in Figure 4.

Limitations and Future Work

  • Computational Cost of K-means: While V-Kmeans was tested, the paper notes that K-means is computationally expensive and doesn't guarantee global optimization.
  • Future Applications: The authors suggest that PCT is a general tool that can be applied to other tasks in autonomous driving, such as 3D object detection and compression, beyond its demonstrated use in representation and reconstruction.
  • Decoder Choice: While fully-connected layers performed better in their experiments, exploring other decoder architectures could be a future direction.

Summary updated

Abstract

We present a novel graph-neural-network-based system to effectively represent large-scale 3D point clouds with the applications to autonomous driving. Many previous works studied the representations of 3D point clouds based on two approaches, voxelization, which causes discretization errors and learning, which is hard to capture huge variations in large-scale scenarios. In this work, we combine voxelization and learning: we discretize the 3D space into voxels and propose novel graph inception networks to represent 3D points in each voxel. This combination makes the system avoid discretization errors and work for large-scale scenarios. The entire system for large-scale 3D point clouds acts like the blocked discrete cosine transform for 2D images; we thus call it the point cloud neural transform (PCT). We further apply the proposed PCT to represent real-time LiDAR sweeps produced by self-driving cars and the PCT with graph inception networks significantly outperforms its competitors.
BibTeX
@inproceedings{chen2019pct,
  author = {Siheng Chen and Sufeng Niu and Tian Lan and Baoan Liu},
  title = {{PCT: Large-Scale 3d Point Cloud Representations Via Graph Inception Networks with Applications to Autonomous Driving}},
  booktitle = {ICIP},
  year = {2019},
  url = {https://openreview.net/forum?id=Ny271pBUWZ},
}
Figure