HGNet: Learning Hierarchical Geometry from Points, Edges, and Surfaces

Ting Yao, Yehao Li, Yingwei Pan, Tao MeiCVPR 2023 · DOI

Introduction

3D point clouds are fundamentally unstructured data. Unlike the regular grid of pixels in 2D images, points in 3D space lack a fixed topology. Existing deep learning methods often treat point clouds as independent sets of coordinates or focus on limited local neighborhoods. However, these approaches frequently overlook the inherent compositional nature of 3D geometry. In the physical world, points constitute edges, edges define surfaces, and adjacent surfaces form larger "super-surfaces."

HGNet (Hierarchical Geometry Network), presented at CVPR 2023, addresses this by explicitly modeling a four-level geometric hierarchy. By moving beyond simple point-wise operations, HGNet captures multi-scale geometric structures, leading to a more robust understanding of 3D shapes.

The four-level geometric hierarchy constructed by HGNet: points, edges, surfaces, and super-surfaces.

Methodology

The core innovation of HGNet lies in its structured approach to feature learning, involving bottom-up hierarchy construction and top-down feature aggregation.

Hierarchy Construction

The model builds a geometric hierarchy in a bottom-up fashion:

  1. Points: The raw input 3D coordinates.
  2. Edges: Defined by connecting a point to its nearest neighbors.
  3. Surfaces: Formed by connecting edges to additional neighboring points to create triangles.
  4. Super-surfaces: Clusters of adjacent surfaces that share common edges.

Feature Aggregation and the HG Block

Unlike traditional models that only aggregate features from local to global scales, HGNet employs a top-down enrichment process. Information from super-surfaces is propagated down to surfaces, then to edges, and finally to points. This ensures that point-level representations are contextually aware of the broader geometric structures they inhabit.

The HGNet architecture featuring the HG Block and HG Embedding modules for multi-granular feature integration.

The Hierarchical Geometry (HG) block is the central component of the architecture. It integrates embeddings from points, edges, and surfaces using a Transformer-based self-attention mechanism. This allows the network to dynamically weigh different geometric primitives, unifying them into a holistic local descriptor. The overall architecture follows a multi-stage U-Net style design, utilizing Farthest Point Sampling (FPS) for efficient downsampling.

Experimental Results

HGNet was evaluated across several rigorous 3D benchmarks, demonstrating state-of-the-art performance:

  • ScanObjectNN (Classification): On this challenging real-world dataset characterized by noise and occlusions, HGNet achieved 89.2% Overall Accuracy (OA), a significant 1.5% improvement over previous benchmarks like PointNeXt-S.
  • ModelNet40: The model reached 94.5% OA on synthetic CAD models, proving its versatility across data types.
  • PartNet (Segmentation): HGNet achieved 55.6% mIoU, outperforming the PosPool baseline by 1.8%.
  • S3DIS (Semantic Segmentation): For large-scale indoor scenes, the model reached 70.8% mIoU (Area-5).

Ablation studies confirmed that the inclusion of all three levels (points, edges, and surfaces) is crucial, as they provide complementary geometric information that a single primitive cannot capture alone.

Conclusion

HGNet demonstrates that explicitly modeling the compositional hierarchy of 3D shapes provides a powerful inductive bias for point cloud analysis. By enriching point features with higher-order geometric context through a top-down approach, the model achieves superior performance on both synthetic and real-world datasets. While the added complexity of hierarchy construction is a consideration, the gains in accuracy and structural understanding represent a significant step forward in 3D deep learning.

Summary updated

BibTeX
@inproceedings{yao2023hgnet,
  author = {Ting Yao and Yehao Li and Yingwei Pan and Tao Mei},
  title = {{HGNet: Learning Hierarchical Geometry from Points, Edges, and Surfaces}},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year = {2023},
  doi = {10.1109/cvpr52729.2023.02092},
}
Figure