DensePoint: Learning Densely Contextual Representation for Efficient Point Cloud Processing

Yongcheng Liu, Bin Fan, Gaofeng Meng, Jiwen Lu, Shiming Xiang, Chunhong Pan

Abstract

DensePoint is a novel deep learning architecture designed to bring the efficiency of dense connections to the processing of irregular 3D point clouds. By repeatedly aggregating multi-level and multi-scale features through a generalized point convolution (PConv) operator, DensePoint achieves state-of-the-art accuracy on shape classification and segmentation tasks. Notably, the architecture is significantly more lightweight and faster than previous hierarchical models, offering a highly efficient solution for real-world 3D data processing.

Introduction

Processing 3D point clouds presents a unique challenge because, unlike 2D images, points are unstructured and do not reside on a regular grid. Early architectures like PointNet treated points independently, failing to capture local geometric patterns. Subsequent models like PointNet++ introduced hierarchical grouping to address this, but often relied on computationally expensive multi-scale grouping that did not fully exploit the relationships between different semantic levels.

Contextual disambiguation: sufficient context allows the model to distinguish a bottle from a vase, whereas local patches might be ambiguous.

The core issue identified by the authors is context. Without sufficient surrounding information, a local patch of a "bottle" can appear identical to a "vase." DensePoint addresses this by capturing "densely contextual" information—integrating low-level geometric details with high-level semantic concepts—without a prohibitive increase in computational cost.

Methodology

DensePoint is built upon three primary pillars designed for efficiency and representational power:

  1. Generalized Convolution (PConv): To handle unstructured data, the authors define a "Point Convolution" consisting of a feature transformation via a shared MLP followed by a symmetric aggregation function (e.g., max pooling). This ensures the operation is permutation invariant.
  2. Dense Connection Mode: Inspired by DenseNet, DensePoint connects every layer to every subsequent layer. The $L$-th layer receives the concatenated feature maps of all preceding layers, allowing the network to reuse features and organically build a hierarchy of multi-scale context.
  3. Efficient "Narrow" Architecture: To prevent parameter explosion from dense connections, the model uses a small growth rate ($k=24$). To maintain expressive power, the authors introduce Enhanced PConv (ePConv), which utilizes filter grouping to increase channel count without a proportional rise in FLOPs.
DensePoint architecture: PConv layers and the dense connection mode facilitate organic receptive field growth and feature reuse.

Key Results and Insights

DensePoint demonstrates that dense connections allow for organic multi-scale learning. Instead of manually defining scales at each layer, the network naturally aggregates different scales as it deepens. This leads to several key performance benchmarks:

  • Classification Accuracy: The model achieved 93.2% accuracy on ModelNet40, surpassing PointNet++ (90.7%).
  • Extreme Efficiency: A lightweight 6-layer version of DensePoint achieved 92.1% accuracy with only 0.53M parameters and 148M FLOPs, making it over 11x more efficient than PointNet++ (1684M FLOPs).
  • Segmentation & Normal Estimation: DensePoint set a new state-of-the-art on the ShapeNet part benchmark (84.2% mIoU) and reduced normal estimation error by 48.6% compared to PointNet++.
  • Robustness: The feature reuse mechanism makes the model remarkably robust to noise and random point displacement.

Conclusion and Future Work

While DensePoint offers significant improvements in efficiency and accuracy, the authors note limitations regarding variations in point density, which are common in LiDAR scans. Future work may involve incorporating attention mechanisms to weight different semantic levels during concatenation rather than treating them uniformly. Additionally, while the model excels at general shapes, it still faces challenges with highly intricate, thin structures like curtains or plants in fine-grained tasks like normal estimation.

Summary updated

Abstract

Point cloud processing is very challenging, as the diverse shapes formed by irregular points are often indistinguishable. A thorough grasp of the elusive shape requires sufficiently contextual semantic information, yet few works devote to this. Here we propose DensePoint, a general architecture to learn densely contextual representation for point cloud processing. Technically, it extends regular grid CNN to irregular point configuration by generalizing a convolution operator, which holds the permutation invariance of points, and achieves efficient inductive learning of local patterns. Architecturally, it finds inspiration from dense connection mode, to repeatedly aggregate multi-level and multi-scale semantics in a deep hierarchy. As a result, densely contextual information along with rich semantics, can be acquired by DensePoint in an organic manner, making it highly effective. Extensive experiments on challenging benchmarks across four tasks, as well as thorough model analysis, verify DensePoint achieves the state of the arts.
BibTeX
@inproceedings{liu2019densepoint,
  author = {Yongcheng Liu and Bin Fan and Gaofeng Meng and Jiwen Lu and Shiming Xiang and Chunhong Pan},
  title = {{DensePoint: Learning Densely Contextual Representation for Efficient Point Cloud Processing}},
  booktitle = {IEEE/CVF International Conference on Computer Vision},
  year = {2019},
  doi = {10.1109/iccv.2019.00534},
}
Figure