Spatially Invariant Unsupervised 3D Object Segmentation with Graph Neural Networks

Tianyu Wang, Miaomiao Liu, Kee Siong Ng

Introduction

Human perception naturally decomposes complex environments into distinct objects. For artificial intelligence to interact effectively with the physical world—such as a robot navigating a cluttered room—it requires a similar object-centric understanding. While unsupervised object discovery has matured in 2D image processing, 3D point clouds present unique challenges: irregular data structures, varying point densities, and a lack of natural correspondence between input and generated points.

The SPAIR3D Framework

The authors propose SPAIR3D, a Variational Autoencoder (VAE) framework designed to decompose 3D scenes into a spatial mixture of objects without human labels. The system achieves spatial invariance by dividing the scene into a voxel grid. Each non-empty cell proposes a local "object glimpse," allowing the model to learn object features independently of their global position.

The SPAIR3D pipeline: A voxel grid decomposes the scene into local glimpses for object-centric reconstruction.

Core Technical Contributions

Two primary innovations drive SPAIR3D's performance:

  1. Chamfer Mixture Loss (CML): Standard Chamfer Distance lacks the probabilistic weights necessary for a mixture model. CML provides a mathematical derivation that allows the model to assign points to specific object components or the background during training.
  2. Point Graph Decoder (PGD): To handle varying point counts, the decoder utilizes Graph Neural Networks (PointGNN). This approach treats point cloud generation as a diffusion-like process, enabling dynamic reconstruction of objects.
Detailed glimpse processing: The model uses PointGNNs and a Point Graph Flow for object reconstruction and mask generation.

Experimental Results

SPAIR3D was evaluated on synthetic datasets (Unity Object Room/Table) and real-world data (S3DIS). Key findings include:

  • High Accuracy: On synthetic data, SPAIR3D achieved an Adjusted Rand Index (ARI) of ~0.91, nearing the performance of supervised baselines (0.97).
  • Zero-Shot Scalability: Models trained on scenes with 2–5 objects successfully segmented scenes with 16 objects without further training.
  • Real-World Robustness: The model outperformed traditional unsupervised methods like Mean-shift on the S3DIS dataset, particularly in dense environments with clustered furniture.

Conclusion and Future Work

SPAIR3D demonstrates that geometric structure alone is sufficient for learning "objectness" in 3D. While highly effective, current limitations include the "one cell, one object" assumption and potential fragmentation of very large objects. Future iterations may explore hierarchical models or memory mechanisms to enhance temporal consistency and scale handling.

Summary updated

BibTeX
@inproceedings{wang2022spatially,
  author = {Tianyu Wang and Miaomiao Liu and Kee Siong Ng},
  title = {{Spatially Invariant Unsupervised 3D Object Segmentation with Graph Neural Networks}},
  booktitle = {European Conference on Computer Vision},
  year = {2022},
  doi = {10.1007/978-3-031-20050-2\_8},
}
Figure