Abstract
Unsupervised feature learning for 3D point clouds is a critical challenge in computer vision, aiming to extract meaningful representations without extensive manual labeling. Traditional methods often rely on global self-reconstruction, which may neglect fine-grained local geometry. We propose the Multi-Angle Point Cloud Variational Auto-Encoder (MAP-VAE), which leverages a novel "half-to-half" prediction task across multiple viewpoints. By splitting a point cloud into front and back halves using geodesic distances and training the network to predict the hidden portions, MAP-VAE learns a comprehensive representation of 3D shapes. Our approach demonstrates superior performance on classification and segmentation benchmarks, rivaling supervised methods in some categories.
Introduction
Supervised learning has driven significant progress in 3D point cloud analysis for applications like autonomous driving and robotics. However, the dependency on large-scale labeled datasets remains a bottleneck. Unsupervised learning offers a path forward, but existing global reconstruction-based methods often fail to capture subtle local features. To address this, we introduce a framework that forces the model to understand the spatial relationships between different parts of a shape by predicting missing geometry from multiple angles, effectively playing a game of "geometric hide-and-seek."
Methodology
The MAP-VAE framework is built around three primary components: multi-angle splitting, feature aggregation, and a dual-task output consisting of reconstruction and prediction.
Multi-Angle Geodesic Splitting
Unlike simple Euclidean splitting, which can produce fragmented patches, we utilize geodesic distance (distance along the surface) to partition the point cloud into a "front half" (visible) and a "back half" (hidden) from various viewpoints. This ensures that the halves are semantically and geometrically continuous, facilitating more effective learning of local structures.
Model Architecture
The architecture consists of three specialized branches:
- Aggregation Branch (A): This branch uses a global encoder for the full shape and a local encoder for the sequence of front halves. An Aggregation RNN then processes these features to capture spatial dependencies across the different views.
- Prediction Branch (P): This implements the "half-to-half" task. The model takes the features of the front halves and attempts to predict the features of the corresponding hidden back halves, requiring a deep understanding of 3D geometry.
- Reconstruction Branch (R): A standard VAE component that ensures the model can reconstruct the entire original point cloud from its learned latent space, facilitating shape generation.
Experimental Evaluation
We evaluated MAP-VAE on the gold standards of 3D deep learning: ModelNet10/40 and the ShapeNet Part dataset.
- Classification: MAP-VAE achieved a state-of-the-art accuracy of 90.15% on ModelNet40 among unsupervised methods.
- Segmentation: The model significantly outperformed competitors in part segmentation across almost all categories (e.g., Aero, Chair, Car), proving that its learned features capture intricate local details better than global-only methods.
- Shape Completion: Qualitative results demonstrate the model's ability to "fill in the blanks" for partial point clouds, producing sharper edges and more accurate geometry.
Key Insights
The success of MAP-VAE stems from the realization that predicting missing parts is a more challenging and semantically rich task than simple compression. By enforcing a variational constraint, the model learns a smooth, continuous latent space suitable for both analysis and generation. Furthermore, the use of geodesic distance is crucial for maintaining the structural integrity of the input halves during training, preventing the "shattered" patches common in Euclidean-based splitting.
Conclusion
MAP-VAE provides a robust framework for unsupervised 3D feature learning. While the use of RNNs and multiple viewpoints (typically $W=6$) increases computational complexity, the gains in feature quality are significant. Future work may involve optimizing viewpoint selection and scaling the approach to large-scale real-world LiDAR data from autonomous vehicles.
Summary updated