Introduction
Hierarchical architectures are fundamental to deep learning on 3D point clouds, enabling models to capture features across multiple spatial scales. Traditionally, these architectures rely on heuristic operations: Farthest Point Sampling (FPS) to select representative points and K-Nearest Neighbors (KNN) to group them. However, these methods suffer from two primary drawbacks. First, they are task-irrelevant, relying solely on Euclidean distance which may fail to capture semantic relationships between distant parts of an object (e.g., the wings and fuselage of an airplane). Second, they are computationally inefficient, involving expensive search operations that hinder real-time inference.
To address these issues, this paper introduces the Dynamic Points Agglomeration Module (DPAM). DPAM replaces fixed heuristics with a learnable process that clusters points in semantic space, allowing for more flexible and efficient hierarchical feature extraction.
The Dynamic Points Agglomeration Module (DPAM)
The core innovation of this work is treating the point cloud as a graph where relationships are learned rather than fixed. DPAM takes a feature matrix and a similarity graph as input and utilizes Graph Convolutional Network (GCN) layers to generate a "soft" agglomeration matrix ($S$).
Learnable Agglomeration
Unlike KNN, which selects a hard set of neighbors, the agglomeration matrix $S$ defines weights for how every input point contributes to a reduced set of output points. This allows the model to group points that are functionally related, even if they are physically distant. The entire process of sampling, grouping, and pooling is consolidated into a single, highly efficient matrix multiplication:
$$F_{out} = S^T F_{in}$$
This shift from iterative searching to linear algebra significantly accelerates processing on modern GPUs.
Architecture and Scalability
To manage the memory requirements of large point clouds, the authors implement a parameter-sharing scheme. The point cloud is divided into segments processed by a shared DPAM. As the network deepens, these segments are merged, effectively increasing the receptive field. For segmentation tasks, the model employs a U-Net style structure, using the transpose of the learned agglomeration matrix for symmetric up-sampling.
Experimental Results
The DPAM-based model was evaluated on several standard 3D benchmarks, including ModelNet40 for classification and ShapeNetPart/S3DIS for segmentation.
- State-of-the-Art Accuracy: The method achieved 91.9% accuracy on ModelNet40, surpassing the baseline PointNet++ (90.7%) and competing with other top-tier point-based methods.
- Inference Speed: One of the most significant gains was in efficiency. In inference tests on ModelNet40, DPAM required only 36.6 ms, a nearly 4.5x speedup over PointNet++'s 163.2 ms.
- Robustness to Noise: The learnable nature of DPAM provides superior resilience to data corruption. When tested with Gaussian noise (replacing 500 points), DPAM maintained significantly higher accuracy compared to traditional heuristic-based models.
Conclusion
Dynamic Points Agglomeration represents a significant step toward making 3D point cloud networks more adaptive and efficient. By moving away from Euclidean-fixed heuristics toward learnable semantic clustering, DPAM achieves superior performance and speed. While the model size is larger than some alternatives, its robustness and inference efficiency make it a compelling choice for real-world 3D applications such as object detection and scene understanding.
Summary updated