Deep Unsupervised Learning of 3D Point Clouds via Graph Topology Inference and Filtering

Siheng Chen, Chaojing Duan, Yaoqing Yang, Duanshun Li, Chen Feng, Dong Tian

TL;DR

This paper introduces a novel deep autoencoder designed to learn compact, representative features of 3D point clouds without any human labels. By combining a "folding" mechanism with learnable graph topology and graph filtering, the model can reconstruct complex 3D shapes with high fidelity, outperforming existing unsupervised methods in both reconstruction and downstream classification tasks.

What Problem Does This Paper Address?

3D point clouds are the "raw" data produced by LiDAR and depth cameras, but they are notoriously difficult for AI to process because they are unorganized, irregular, and sparse.

Most existing methods try to solve this by either:

  1. Voxelization: Converting points into a 3D grid (like pixels), which causes massive memory overhead and "quantization errors" (loss of detail).
  2. Supervised Learning: Requiring massive amounts of human-labeled data, which is expensive and slow to produce.

The authors aim to create an unsupervised system that handles raw points directly. The core challenge is designing a decoder that can take a compressed "codeword" and accurately reconstruct the intricate geometric details of a 3D object without being told what the object is.

Methods

The authors propose an encoder-decoder architecture where the magic happens primarily in the decoder.

  • The Encoder: Uses a PointNet-like architecture (cascaded MLPs and max-pooling) to compress a point cloud into a low-dimensional latent code.
  • The Decoder (The Innovation): It consists of three distinct modules:
  • Folding Module: It takes a standard 2D grid (like a flat sheet of paper) and "folds" it into 3D space based on the latent code. This produces a "coarse" reconstruction.
  • Graph-Topology Inference Module: This is the "brain" of the decoder. It learns the pairwise relationships (edges) between points. Instead of assuming points are just nearby in space, it learns a graph that represents the underlying structure of the object.
  • Graph-Filtering Module: This module uses the learned graph to apply a low-pass filter to the coarse reconstruction. Intuitively, it "smooths" the surface and refines the positions of the points to capture fine details like airplane engines or chair legs that simple folding misses.

The model is trained using an Augmented Chamfer Distance, a robust loss function that measures how closely the reconstructed point cloud matches the original.

Datasets

The model was evaluated on several standard 3D benchmarks:

  • ShapeNet / ShapeNetCore: Over 50,000 3D models across 55 categories.
  • ModelNet40 & ModelNet10: Standard datasets for 3D object classification.
  • New Fine-Grained Benchmark: The authors manually labeled subcategories within ModelNet40 (e.g., distinguishing between different types of airplane wing shapes) to test if their unsupervised features could capture subtle geometric differences.

Key Insights and Ideas

  • Graph Smoothness > Spatial Smoothness: The authors prove theoretically that "graph smoothness" (points being similar to their neighbors in a learned graph) is a much more powerful prior for 3D surfaces than simple spatial smoothness. This allows the model to handle "high-order genus" shapes—objects with holes, like a torus or a chair with gaps.
  • Folding has limits: A single 2D sheet can't easily fold into a complex 3D object with holes without "tearing." The graph-topology module effectively allows the network to "cut and glue" the 2D lattice to fit complex topologies.
  • The Decoder as a Teacher: By making the decoder more sophisticated (adding graph inference), the encoder is forced to learn much richer, more descriptive latent features to satisfy the decoder's needs.

Key Results

  • Reconstruction: The proposed method achieved lower reconstruction errors (measured by Chamfer distance) than AtlasNet and 3DCapsNet across almost all categories in ShapeNetCore.
  • Transfer Classification: After training the model on ShapeNet (unsupervised), the authors used the learned features to train a simple linear classifier on ModelNet.
  • On ModelNet40, they achieved 89.67% accuracy, significantly outperforming Latent-GAN (85.7%) and even rivaling some supervised methods.
  • Detail Preservation: Qualitative results show that graph filtering successfully recovers fine details (like engines on a "swept wing" aircraft) that are completely lost in models that only use a folding module.

Limitations and Future Work

  • Computational Complexity: Learning a full graph adjacency matrix involves $O(M^2)$ memory, where $M$ is the number of points. This makes scaling to very high-resolution point clouds (tens of thousands of points) difficult on current hardware.
  • Divide-and-Conquer: The authors suggest that a "divide-and-conquer" strategy—breaking large point clouds into smaller patches—could be a potential solution for the scaling issue.
  • Fixed 2D Lattice: While the graph is learnable, the initial 2D lattice is fixed. Future work could explore more flexible initializations for the folding process.

Summary updated

Abstract

We propose a deep autoencoder with graph topology inference and filtering to achieve compact representations of unorganized 3D point clouds in an unsupervised manner. Many previous works discretize 3D points to voxels and then use lattice-based methods to process and learn 3D spatial information; however, this leads to inevitable discretization errors. In this work, we try to handle raw 3D points without such compromise. The proposed networks follow the autoencoder framework with a focus on designing the decoder. The encoder of the proposed networks adopts similar architectures as in PointNet, which is a well-acknowledged method for supervised learning of 3D point clouds. The decoder of the proposed networks involves three novel modules: the folding module, the graph-topology-inference module, and the graph-filtering module. The folding module folds a canonical 2D lattice to the underlying surface of a 3D point cloud, achieving coarse reconstruction; the graph-topology-inference module learns a graph topology to represent pairwise relationships between 3D points, pushing the latent code to preserve both coordinates and pairwise relationships of points in 3D point clouds; and the graph-filtering module couples the above two modules, refining the coarse reconstruction through a learnt graph topology to obtain the final reconstruction. The proposed decoder leverages a learnable graph topology to push the codeword to preserve representative features and further improve the unsupervised-learning performance. We further provide theoretical analyses of the proposed architecture. We provide an upper bound for the reconstruction loss and further show the superiority of graph smoothness over spatial smoothness as a prior to model 3D point clouds. In the experiments, we validate the proposed networks in three tasks, including 3D point cloud reconstruction, visualization, and transfer classification. The experimental results show that (1) the proposed networks outperform the state-of-the-art methods in various tasks, including reconstruction and transfer classification; (2) a graph topology can be inferred as auxiliary information without specific supervision on graph topology inference; (3) graph filtering refines the reconstruction, leading to better performances; and (4) designing a powerful decoder could improve the unsupervised-learning performance, just like a powerful encoder.
BibTeX
@article{chen2020deep,
  author = {Siheng Chen and Chaojing Duan and Yaoqing Yang and Duanshun Li and Chen Feng and Dong Tian},
  title = {{Deep Unsupervised Learning of 3D Point Clouds via Graph Topology Inference and Filtering}},
  journal = {IEEE Trans. Image Process.},
  year = {2020},
  url = {https://openreview.net/forum?id=97RK3fjNrz8},
}

Loading related content...

Figure