RGL-NET: A Recurrent Graph Learning Framework for Progressive Part Assembly

Abhinav Narayan, Rajendra Nagar, Shanmuganathan RamanWACV 2022

TL;DR

This paper introduces RGL-NET, a novel recurrent graph learning framework for assembling 3D shapes from their constituent parts. By processing parts sequentially and learning inter-part relationships progressively, RGL-NET significantly improves part accuracy and connectivity, outperforming existing methods, especially in handling structural variations.

What Problem Does This Paper Address?

Automated assembly of 3D objects from their parts is a fundamental challenge in robotics and computer vision. While existing methods can assemble objects, they often struggle with structural variations within a category (e.g., different chair designs) and typically process all parts simultaneously. This can lead to confusion, especially when parts have similar appearances (like a chair seat and back), resulting in incorrect placement. The paper aims to develop a generalized framework that can robustly assemble objects with diverse structures without prior semantic knowledge of the global shape.

Methods

The core of RGL-NET is a Recurrent Graph Learning (RGL) framework. It combines a graph learning backbone to model inter-part relationships with a progressive assembly encoder.

  • Graph Learning Backbone: This component models the object as a dynamic graph where nodes represent part features and edges represent their relationships. It uses an attention mechanism to weigh the importance of different part relationships.
  • Progressive Assembly Encoder: This is the novel aspect. Instead of processing all parts at once, RGL-NET uses a bidirectional GRU to encode assembly information sequentially. This means the network learns from parts that have already been "assembled" (processed), allowing it to build context and improve predictions for subsequent parts. This sequential processing is crucial for reducing inter-part confusion and handling structural variations.
  • Noise Injection: To further encourage exploration of structural variations and improve robustness, random noise is injected into the initial hidden states of the GRU.
  • Pose Prediction: The framework predicts the 6D pose (rotation and translation) for each part.

Datasets

The primary dataset used is PartNet, a large-scale dataset with fine-grained part segmentation. The authors focus on the three largest categories: chair, table, and lamp. Each part is represented by a point cloud of 1000 points, which are then transformed into a canonical space using PCA to ensure invariance to rigid transformations.

Key Insights and Ideas

  • Sequential Assembly Matters: The paper strongly advocates for a progressive, step-by-step assembly process, mirroring how humans often assemble objects. This sequential approach significantly reduces confusion between similar parts.
  • Recurrent Encoding for Context: Using a bidirectional GRU to encode the assembly sequence allows the network to build a rich contextual understanding of already placed parts, which directly informs the placement of subsequent parts.
  • Latent Space for Shape Recovery: The learned latent space of the RGL-NET framework can be used for shape recovery from component point clouds, even without access to the global shape structure. This suggests the latent representation captures essential structural information.
  • Noise for Robustness: Injecting random noise into the network's initial states helps it explore a wider range of structural variations and improves its robustness to different object configurations.

Key Results

  • Improved Accuracy: RGL-NET achieves up to 10% improvement in part accuracy and 15% improvement in connectivity accuracy on the PartNet dataset compared to the state-of-the-art (B-DGL [11]).
  • Competitive Performance: The framework shows competitive performance even without the use of random noise, making it faster and more stable.
  • Qualitative Improvements: Visualizations show RGL-NET better handles structurally ambiguous parts (e.g., chair arms and legs) and achieves more accurate predictions for complex connections (e.g., lamp bulb and cover).
  • Shape Recovery: The method demonstrates successful shape recovery from its latent space, providing a coarse but structurally sound outline of the object.
  • Robustness to Missing Parts: The framework shows resilience to missing parts, with accuracy increasing at certain deletion percentages, suggesting it can identify deficiencies.

Limitations and Future Work

  • Optimal Ordering: While the paper identifies top-down ordering as optimal among tested strategies, finding the absolute global optimum for assembly order is computationally challenging and remains an open area.
  • Future Work: The authors suggest developing reordering frameworks to automatically arrange randomly ordered parts. They also propose incorporating constraints like symmetry into the assembly strategy to reduce the search space and improve efficiency. Exploring the latent space for retrieving part-connectivity is another promising direction.

Summary updated

BibTeX
@inproceedings{narayan2022rgl,
  author = {Abhinav Narayan and Rajendra Nagar and Shanmuganathan Raman},
  title = {{RGL-NET: A Recurrent Graph Learning Framework for Progressive Part Assembly}},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
  year = {2022},
}
Figure