Triangle-Net: Towards Robustness in Point Cloud Learning

Chenxi Xiao, Juan P. Wachs

Introduction

Deep learning on 3D point clouds has seen significant progress, yet most state-of-the-art models rely on dense, well-aligned datasets. In practical applications like autonomous driving or robotics, sensors often capture extremely sparse data (fewer than 50 points) with arbitrary rotations. Triangle-Net introduces a "robust-by-design" architecture that leverages geometric invariants to maintain high performance under these challenging conditions.

Methodology

The fundamental shift in Triangle-Net is moving away from absolute $(x, y, z)$ coordinates, which are sensitive to rotation and translation, toward a hypergraph representation based on point triplets.

Invariant Feature Extraction

The model extracts hand-crafted geometric features from triangles formed by triplets of points and their surface normals. These features include relative distances and internal angles, which remain constant regardless of the object's orientation in 3D space. This mathematical invariance eliminates the need for extensive data augmentation for rotation.

Geometric features extracted from point triplets (triangles) ensure rotation and translation invariance.

Triangle-Net Architecture

The extracted features are processed through an end-to-end graph neural network. The architecture employs a mapping function with shortcuts to facilitate deep feature extraction and utilizes multi-task learning to refine latent representations. By simultaneously training on classification, part segmentation, and voxel reconstruction, the model learns a more robust global understanding of the object's geometry.

The Triangle-Net architecture processes sparse point clouds through feature extraction, a mapping function, and multi-task output heads.

Key Results and Discussion

Triangle-Net demonstrates remarkable resilience to data degradation. On the ModelNet40 benchmark, it achieves over 70% accuracy using only 16 points under arbitrary rotation—a scenario where standard models like PointNet fail significantly.

  • Extreme Sparsity: Even with as few as 8 points, the model maintains nearly 50% accuracy, outperforming human-level recognition in similar conditions.
  • Noise Robustness: The model shows graceful degradation when subjected to Gaussian jitter, dropping only 28.5% in accuracy under heavy noise compared to the 58% drop observed in coordinate-based models.
  • Real-World Application: Testing on the ScanObjectNN dataset confirms that the model's performance translates well to noisy, partial real-world scans.

Conclusion

Triangle-Net proves that geometric priors—specifically those derived from point triplets—can dramatically improve the robustness of 3D recognition systems. While it faces computational challenges with very dense data due to the $O(n^3)$ complexity of triangle sampling, its performance on sparse and rotated data sets a new standard for robust point cloud learning.

Summary updated

Abstract

Three dimensional (3D) object recognition is becoming a key desired capability for many computer vision systems such as autonomous vehicles, service robots and surveillance drones to operate more effectively in unstructured environments. These real-time systems require effective classification methods that are robust to various sampling resolutions, noisy measurements, and unconstrained pose configurations. Previous research has shown that points' sparsity, rotation and positional inherent variance can lead to a significant drop in the performance of point cloud based classification techniques. However, neither of them is sufficiently robust to multifactorial variance and significant sparsity. In this regard, we propose a novel approach for 3D classification that can simultaneously achieve invariance towards rotation, positional shift, scaling, and is robust to point sparsity. To this end, we introduce a new feature that utilizes graph structure of point clouds, which can be learned end-to-end with our proposed neural network to acquire a robust latent representation of the 3D object. We show that such latent representations can significantly improve the performance of object classification and retrieval tasks when points are sparse. Further, we show that our approach outperforms PointNet and 3DmFV by 35.0% and 28.1% respectively in ModelNet 40 classification tasks using sparse point clouds of only 16 points under arbitrary SO(3) rotation.
BibTeX
@inproceedings{xiao2021triangle,
  author = {Chenxi Xiao and Juan P. Wachs},
  title = {{Triangle-Net: Towards Robustness in Point Cloud Learning}},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
  year = {2021},
  doi = {10.1109/wacv48630.2021.00087},
}
Figure