Efficient and Feasible Robotic Assembly Sequence Planning via Graph Representation Learning

Matan Atad, Jianxiang Feng, Ismael Rodríguez, Maximilian Durner, Rudolph Triebel

TL;DR

This paper introduces GRACE, a novel graph-based deep learning approach for robotic assembly sequence planning (RASP). GRACE efficiently predicts feasible assembly sequences and can also detect infeasible ones, addressing key challenges in manufacturing automation and product customization.

What Problem Does This Paper Address?

Modern manufacturing is increasingly demanding flexibility and customization, requiring robots to perform complex assembly tasks. A major hurdle is Robotic Assembly Sequence Planning (RASP), which involves determining the correct order of assembly steps. This problem is computationally challenging due to the combinatorial explosion of possible sequences. Furthermore, verifying if a predicted sequence is actually executable by a robot is time-consuming and expensive. Existing methods often struggle with generalization to new product variants and are inefficient in checking feasibility. This paper aims to tackle these issues by developing a more efficient and generalizable RASP method.

Methods

The core of the proposed solution lies in two main components:

  1. Assembly Graph: A novel heterogeneous graph representation for product assemblies. This graph captures detailed spatial relationships by including nodes for both parts and their surfaces, with edges representing geometric relations between surfaces. This representation is more fine-grained than previous approaches.
  2. GRACE (Graph Assembly Processing Network): A deep learning policy architecture based on Graph Neural Networks (GNNs), specifically Graph Attention Networks (GATs). GRACE takes the Assembly Graph as input and predicts assembly sequences in a step-by-step manner. It's designed to be invariant to the number of parts and memory-efficient due to shared weights. GRACE also incorporates a mechanism for predicting the feasibility of an assembly sequence.

The RASP problem is framed as a sequential decision-making problem using a Markov Decision Process (MDP). GRACE is trained using imitation learning on simulated expert demonstrations.

Datasets

The authors generated a synthetic dataset of aluminum assemblies using an in-house simulation software called MediView. These assemblies range in size from 3 to 7 parts. The dataset includes information on feasible assembly sequences and their corresponding placement actions.

Key Insights and Ideas

  • Graph Representation is Key: Representing assemblies as detailed graphs, including surface-level information, is crucial for capturing complex spatial relationships and enabling robust sequence prediction.
  • Generalization is Achievable: The proposed GRACE model demonstrates significant generalization capabilities, performing well on assembly variants not seen during training, even across different sizes.
  • Feasibility Prediction is Essential: Explicitly predicting assembly feasibility is vital for real-world deployment, as it significantly reduces the impact of incorrect predictions and avoids costly robot interventions.
  • Step-by-Step Prediction with Confidence: GRACE predicts probabilities for each potential next step, allowing for a trade-off between precision and recall and enabling the generation of multiple feasible sequences.
  • Early Failure Detection: The analysis reveals that GRACE tends to be overconfident in its predictions for false steps early in the assembly sequence, highlighting an area for improvement.

Key Results

  • High Accuracy in Sequence Prediction: GRACE achieves near-perfect step-by-step AUC scores for intra-sized assemblies and strong complete-sequence AUC scores, especially for smaller assemblies.
  • Effective Generalization: The model shows good performance in inter-sized assembly prediction (many-to-one and one-to-many settings), outperforming previous methods in generalization.
  • Strong Feasibility Prediction: GRACE, using the number of predicted sequences, achieves a high AUC of 0.97 for feasibility prediction. Combining this with other classifiers further improves performance.
  • Efficiency: The model requires only a few milliseconds to infer the next feasible sequence step.

Limitations and Future Work

  • Simulation-Based: The current validation is entirely based on simulation. Real-world deployment will require further testing and adaptation to physical constraints.
  • Simple Geometries: The current success is demonstrated on objects with simple geometries (aluminum profiles). Handling complex 3D objects with curved surfaces remains a future challenge.
  • Bias in Failure Analysis: The failure analysis indicates a bias where GRACE is overconfident in early incorrect predictions, suggesting a need to address imbalances in the training data or model architecture.
  • Future Directions: The authors suggest extending the work to handle more complex 3D objects and incorporating additional constraints like grasp planning for more comprehensive real-world deployment.

Summary updated

Abstract

Automatic Robotic Assembly Sequence Planning (RASP) can significantly improve productivity and resilience in modern manufacturing along with the growing need for greater product customization. One of the main challenges in realizing such automation resides in efficiently finding solutions from a growing number of potential sequences for increasingly complex assemblies. Besides, costly feasibility checks are always required for the robotic system. To address this, we propose a holistic graphical approach including a graph representation called Assembly Graph for product assemblies and a policy architecture, Graph Assembly Processing Network, dubbed GRACE for assembly sequence generation. With GRACE, we are able to extract meaningful information from the graph input and predict assembly sequences in a step-by-step manner. In experiments, we show that our approach can predict feasible assembly sequences across product variants of aluminum profiles based on data collected in simulation of a dual-armed robotic system. We further demonstrate that our method is capable of detecting infeasible assemblies, substantially alleviating the undesirable impacts from false predictions, and hence facilitating real-world deployment soon. Code and training data are available at https://github.com/DLR-RM/GRACE.
BibTeX
@inproceedings{atad2023efficient,
  author = {Matan Atad and Jianxiang Feng and Ismael Rodríguez and Maximilian Durner and Rudolph Triebel},
  title = {{Efficient and Feasible Robotic Assembly Sequence Planning via Graph Representation Learning}},
  booktitle = {IEEE/RSJ International Conference on Intelligent Robots and Systems},
  year = {2023},
  url = {https://openreview.net/forum?id=8TA3oGSuur},
}
Figure