Abstract
The Hierarchical Point-Edge Interaction Network (HPEIN) addresses the challenges of 3D point cloud semantic segmentation by explicitly modeling the relationships between neighboring points. Unlike traditional methods that often discard relational context through simple aggregation, HPEIN utilizes a dual-branch architecture that facilitates bidirectional message passing between point and edge features across multiple scales. By incorporating an auxiliary edge loss to enforce semantic consistency, the model achieves superior performance on major benchmarks, particularly in defining object boundaries.
The Challenge of Point Cloud Segmentation
Semantic segmentation in 3D—assigning category labels like "chair" or "wall" to every point—is inherently difficult due to the irregular, sparse, and unordered nature of point clouds. While early architectures like PointNet++ introduced hierarchical grouping, they often rely on max-pooling to aggregate local information, which can lose the nuanced relational data between points. To truly understand a scene, a network must model whether connected points belong to the same object, a concept the authors term "semantic consistency."
Methodology: Hierarchical Point-Edge Interaction
The researchers propose a two-branch framework that interacts at every level of the network hierarchy.
Dual-Branch Architecture
The core of HPEIN is the interaction between two distinct branches:
- The Point Branch: An encoder-decoder structure that extracts and upsamples point-wise features.
- The Edge Branch: A novel hierarchical branch that constructs a graph of the scene, starting at a coarse resolution and upsampling edges as point resolution increases.
Hierarchical Interaction and Edge Upsampling
Instead of reconstructing a graph from scratch at every layer, HPEIN uses an Edge Upsampling scheme. Edge features from coarser layers are propagated to finer ones using spatial distance-based interpolation. This allows for long-range message passing without the computational burden of full graph construction at high resolutions.
At each layer, point features are fed into the edge branch to generate edge features, while edge features are max-pooled and returned to the point branch to provide "regional guidance" for point-wise predictions.
Auxiliary Edge Loss
To ensure edge features capture meaningful semantic boundaries, the model is trained with an auxiliary task: predicting whether an edge connects points of the same category (intra-class) or different categories (inter-class). This forces the network to learn more discriminative features specifically at the borders between objects.
Key Insights
- Edges as Descriptors: Edges are treated as distinct features capturing local geometric and semantic "texture," rather than just paths for convolution.
- Efficiency through Hierarchy: Hierarchical graph construction and edge upsampling enable the model to handle large point clouds with manageable memory overhead.
- Boundary Regularization: The auxiliary edge loss acts as a powerful regularizer, sharpening the network's ability to distinguish between adjacent objects.
Evaluation and Results
HPEIN was evaluated on the S3DIS and ScanNet v2 datasets, setting new benchmarks at the time of publication:
- S3DIS (Area 5): Achieved an mIoU of 61.85%, significantly outperforming previous state-of-the-art models like PCCN and PointCNN.
- ScanNet v2: Reached an mIoU of 61.8%, a substantial improvement over PointNet++ (33.9%) and PointConv (55.6%).
Qualitative analysis confirms that the model is particularly effective at "cleaning up" semantic boundaries and identifying small, detailed structures that are often blurred by other methods.
Conclusion
By elevating edges to "first-class citizens" and modeling their interaction with point features hierarchically, HPEIN provides a robust solution for 3D semantic segmentation. While the current localized graph construction offers efficiency, future work may explore multi-range edge connections to capture even broader global contexts.
Summary updated