Abstract
Point cloud understanding is a critical task in 3D computer vision, yet the unstructured nature of point sets poses significant challenges for traditional deep learning architectures. We introduce the L2G Auto-encoder (L2G-AE), an unsupervised framework designed to learn robust geometric representations through a local-to-global reconstruction task. By employing a hierarchical self-attention mechanism in the encoder and an RNN-based progressive decoder, the model effectively captures multi-scale local features and their global relationships. L2G-AE achieves state-of-the-art performance on unsupervised classification and shape retrieval benchmarks, demonstrating the efficacy of bottom-up reconstruction for 3D shape understanding.
Introduction
Point clouds, typically represented as sets of $(x, y, z)$ coordinates, are the primary output of 3D sensors like LiDAR. Unlike 2D images, their lack of a regular grid structure makes standard convolutional operations inapplicable. While architectures like PointNet have advanced the field, many existing Auto-encoders (AEs) focus predominantly on global structure, often neglecting the subtle local geometric features essential for fine-grained shape understanding. To address this, we propose a "Local-to-Global" (L2G) approach that emphasizes bottom-up reconstruction, forcing the model to understand how local patches compose the global form.
Methods
The L2G-AE architecture consists of a hierarchical self-attention encoder and a progressive RNN-based decoder, connected by a global feature vector.
Hierarchical Self-Attention Encoder
The encoder moves beyond simple max-pooling by employing a three-level attention mechanism to dynamically weight geometric features:
- Multi-scale Establishment: Centroids are sampled across the shape, and neighboring points are grouped into four distinct scales (neighborhood sizes).
- Point-level Attention: Captures relationships between points within a single patch.
- Scale-level Attention: Dynamically weights the contributions of different neighborhood sizes.
- Region-level Attention: Models the spatial relationships between different parts of the object.
Progressive RNN-based Decoder
The decoder reconstructs the shape by treating multi-scale local areas as a sequence, allowing the model to learn spatial relationships between a point and its increasingly larger neighborhoods.
- Interpolation: The 1024-dimensional global feature is interpolated back to local region centroids using distance-based weighting.
- RNN Decoding: An LSTM-based RNN processes the multi-scale features as a sequence, learning the evolution of geometry from local patches to larger regions.
- Joint Loss: Training is guided by a combination of local and global reconstruction losses, calculated using the Chamfer Distance metric to compare point set similarity.
Results and Discussion
L2G-AE was evaluated on standard benchmarks including ModelNet10, ModelNet40, and ShapeNet55.
- Unsupervised Classification: On ModelNet40, L2G-AE achieved 90.64% accuracy in an unsupervised setting. This significantly outperforms previous unsupervised leaders like FoldingNet (88.4%) and even exceeds the performance of the original supervised PointNet (89.2%).
- Shape Retrieval: In retrieval tasks, L2G-AE achieved a mean Average Precision (mAP) of 67.81%, a substantial improvement over the previous state-of-the-art of 53.42%.
- Point Cloud Upsampling: The model demonstrated the ability to take sparse inputs (625 points) and reconstruct dense, smooth outputs (10,000 points), performing competitively with specialized supervised upsampling networks like PU-Net.
The success of L2G-AE highlights that self-attention allows for a "soft" selection of features, capturing correlations that standard max-pooling ignores. Furthermore, treating geometry as a sequence for RNN decoding provides a powerful inductive bias for understanding spatial hierarchies.
Conclusion
The L2G Auto-encoder provides a robust unsupervised framework for point cloud representation learning. By focusing on local-to-global reconstruction, it captures rich geometric features that generalize well to various downstream tasks. Future work will explore scaling this approach to complex 3D scenes and optimizing the computational overhead associated with hierarchical attention and RNNs.
Summary updated