GaIA: Graphical Information Gain based Attention Network for Weakly Supervised Point Cloud Semantic Segmentation

Min Seok Lee, Seok Woo Yang, Sung Won Han

Semantic segmentation of 3D point clouds is a cornerstone of autonomous driving and robotics. However, traditional methods rely on "dense" labels, requiring human annotators to categorize every single point in a scene—a process that is both labor-intensive and prohibitively slow. GaIA (Graphical Information Gain based Attention Network) addresses this by enabling high-performance segmentation using only sparse supervision, such as a single labeled point per class.

The Challenge of Sparse Supervision

In weakly supervised learning, models are trained on a tiny fraction of data (e.g., 1% annotation). This sparsity often leads to high epistemic uncertainty, where the model produces "fuzzy" feature representations and struggles to define clear boundaries between objects. Existing approaches frequently treat all unlabeled points with equal weight, failing to leverage the model's own internal confidence to improve learning.

Methodology: GI Attention and ArcPoint Loss

GaIA introduces two core innovations integrated into a 3D U-Net architecture, supported by a Siamese branch for consistency:

  1. Graphical Information Gain (GI): This specialized attention mechanism calculates Shannon entropy for every point to quantify uncertainty. It identifies "reliable" points—those with lower entropy than their spatial neighbors—and uses this relative entropy to propagate credible information from certain regions to uncertain ones, sharpening decision boundaries.
  2. ArcPoint Loss: Moving beyond standard Softmax, ArcPoint is inspired by angular margin losses used in facial recognition. It treats labeled points as "anchors" on a hypersphere. For unlabeled points with high entropy, the loss applies an additive angular margin to "push" them toward the most semantically similar labeled anchor, enforcing discriminative and compact feature clusters.
The GaIA framework integrates Entropy Blocks (EB) within a 3D U-Net and utilizes a Siamese branch to calculate ArcPoint loss based on angular margins.

Key Insights: Uncertainty as a Signal

A central philosophy of GaIA is that quantifying what the model doesn't know is the key to learning from sparse data. By recognizing that a point’s reliability is relative to its local 3D context, the GI module effectively filters noise. Furthermore, the use of a hypersphere for feature representation ensures intra-class compactness, which is critical when labels are too few to define class boundaries through standard cross-entropy alone.

Visual comparison of model predictions and entropy maps. Higher entropy (warmer colors) typically correlates with regions of prediction error or boundary ambiguity.

Experimental Results

The framework was evaluated on major benchmarks under extreme sparsity constraints:

  • S3DIS: With only 1% annotation, GaIA achieved 66.5% mIoU, outperforming the previous state-of-the-art (HybridCR) by 1.2%. Even with just one point per class, it maintained a robust 53.7% mIoU.
  • ScanNet-v2: Under 1% annotation, GaIA reached 65.2% mIoU, representing a significant 8.4% improvement over HybridCR.
  • Qualitative Performance: Visualizations demonstrate that GaIA excels at identifying thin or complex structures, such as door frames and beams, which baseline models often blur into surrounding walls.

Limitations and Future Directions

While GaIA sets a new standard for weakly supervised segmentation, it faces certain trade-offs. The inclusion of entropy blocks in every decoder layer can impact computational efficiency; thus, they are strategically placed in the encoder and early decoder stages. Additionally, the model occasionally suffers from being "confidently wrong"—producing low-entropy but incorrect predictions. Future research will focus on targeting these specific failure modes to further bridge the gap between weakly and fully supervised performance.

Summary updated

BibTeX
@inproceedings{lee2023gaia,
  author = {Min Seok Lee and Seok Woo Yang and Sung Won Han},
  title = {{GaIA: Graphical Information Gain based Attention Network for Weakly Supervised Point Cloud Semantic Segmentation}},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
  year = {2023},
  doi = {10.1109/wacv56688.2023.00065},
}
Figure