Semantic Segmentation for Point Cloud Scenes via Dilated Graph Feature Aggregation and Pyramid Decoders

Yongqiang Mao, Xian Sun, Wenhui Diao, Kaiqiang Chen, Zonghao Guo, Xiaonan Lu, Kun FuarXiv 2022 · DOI · arXiv

Abstract

Semantic segmentation of 3D point clouds is challenging due to the structural similarity of different objects at local scales. Current methods often suffer from a "unicity of receptive field," where a fixed scale of neighbor search leads to misclassification. This paper introduces DGFA-Net, a novel architecture that utilizes Dilated Graph Feature Aggregation (DGFA) and a pyramid of decoders to capture multi-scale contextual information. By aggregating features across various receptive fields, DGFA-Net significantly improves segmentation accuracy, particularly for objects with similar local geometries, setting new benchmarks on the S3DIS and Toronto-3D datasets.

Introduction: The Receptive Field Challenge

Assigning category labels to every point in a 3D scene requires understanding both local geometry and global context. Most existing models utilize K-Nearest Neighbor (KNN) graphs to capture local structures. However, these models typically rely on a fixed number of neighbors or a fixed search radius, resulting in a single, static receptive field.

This "unicity of the receptive field" prevents the network from distinguishing between objects that appear identical at a specific scale. For instance, a "board" mounted on a "wall" may be indistinguishable from the wall itself if the network's view is too narrow. Without broader context, the model lacks the information necessary to identify object boundaries accurately.

Figure 1: Comparison of receptive fields. Standard models (top) with a single receptive field often misclassify objects like boards as walls. DGFA-Net (bottom) aggregates multiple receptive fields to achieve correct classification.

DGFA-Net Architecture

To overcome these limitations, the authors propose DGFA-Net, which integrates two key components: Dilated Graph Feature Aggregation and Pyramid Decoders.

Dilated Graph Feature Aggregation (DGFA)

Inspired by dilated convolutions in 2D imaging, the Dilated Graph Convolution (DGConv) employs a "Sparse-KNN Search" strategy. Instead of sampling immediate neighbors, it skips points at specific dilation rates to expand the receptive field without increasing computational overhead. These layers are densely connected to fuse features from multiple scales, allowing the network to perceive both fine details and broad context simultaneously.

Pyramid Decoders and Multi-basis Aggregation Loss

The architecture employs a pyramid of decoders that generate predictions at four resolution levels ($N, N/4, N/16, N/32$). To supervise this multi-scale learning, a Multi-basis Aggregation Loss (MALoss) is introduced. This loss function penalizes errors at every resolution, forcing the network to learn discriminative features across all "zoom levels," from individual points to entire rooms.

Figure 2: The overall framework of DGFA-Net, showcasing the hierarchical graph generation, the DGFA module for multi-scale feature fusion, and the pyramid decoders for multi-resolution supervision.

Experimental Evaluation

DGFA-Net was evaluated on three major benchmarks:

  • S3DIS: An indoor dataset where DGFA-Net achieved a 65.8% mIoU, a 4.0% improvement over the baseline. Notably, it outperformed the baseline by 14% in the "board" category.
  • Toronto-3D: An outdoor LiDAR dataset where the model reached 64.25% mIoU and 94.78% overall accuracy, showing significant gains in identifying road-level objects.
  • ShapeNetPart: Demonstrated robust performance in part-level object segmentation.

Key Insights

  • Biological Inspiration: The model mimics the "Contrast Phenomenon" in human vision, where adjacent stimuli enhance object recognition.
  • Efficiency: Despite its performance gains, DGFA-Net is highly efficient. It achieves higher accuracy than complex models like PointCNN while maintaining a significantly lower parameter count.
  • Robustness through Sparsity: Sparse-KNN search provides the necessary global context for large-scale scenes without the memory burden of dense neighbor searches.

Conclusion

DGFA-Net effectively addresses the receptive field limitations of traditional graph-based point cloud networks. By combining dilated graph convolutions with multi-resolution supervision, it provides a robust and efficient solution for 3D semantic segmentation. Future work may involve scaling the architecture for even larger environments or integrating these multi-scale strategies into Transformer-based models.

Summary updated

BibTeX
@misc{mao2022semantic,
  author = {Yongqiang Mao and Xian Sun and Wenhui Diao and Kaiqiang Chen and Zonghao Guo and Xiaonan Lu and Kun Fu},
  title = {{Semantic Segmentation for Point Cloud Scenes via Dilated Graph Feature Aggregation and Pyramid Decoders}},
  year = {2022},
  eprint = {2204.04944},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2204.04944},
}
Figure