SCANet: Correcting LEGO Assembly Errors with Self-Correct Assembly Network

Yuxuan Wan, Kaichen Zhou, Jinhong Chen, Hao Dong

Introduction

Autonomous robots often struggle with long-term assembly tasks because small mistakes early in the process tend to snowball into total failure. While humans can naturally spot a misplaced LEGO brick and adjust it, robots typically follow a rigid, "blind" sequence of instructions. This paper introduces SCANet, a "Self-Correct Assembly Network" designed to identify and rectify misassembled components by comparing the current state of a build against the provided manual.

The Problem: Error Propagation

The authors define a new challenge: the Single-Step Assembly Error Correction Task. In traditional robotic assembly, if a part is misplaced in an early step, the robot continues as if the state is correct, leading to massive structural discrepancies in later stages. Existing models like MEPNet focus on the assembly itself but lack a mechanism to "double-check" their work.

A comparison of a normal assembly sequence versus one with significant assembly deviations, illustrating how early errors propagate and lead to total failure by step 27.

Methods

SCANet is a framework inspired by the DETR (Detection Transformer) architecture. Its workflow is divided into two main modules:

  1. Convolutional Neural Network Backbone: This module uses a dual-branch "Hourglass" network. One branch processes the "ideal" state from the manual, and the other processes the "actual" (potentially erroneous) assembly result. By comparing these, the network generates a Difference Feature Map ($f_{diff}$) that highlights discrepancies.
  2. Assembly Correction Module: This module treats each assembled component as a query. A Component Pose Encoder integrates 3D voxel data (geometry), the current 6D pose, and 2D image data (color/texture). These queries interact with the Difference Feature Map via a Transformer encoder/decoder to determine if a part is misplaced.

If an error is detected, a Component Pose Corrector classifies the error type and predicts the necessary 3D coordinate and rotation adjustments. Notably, the authors treat pose correction as a classification task rather than regression, which they found made the network more stable and accurate.

Datasets and Evaluation

The researchers introduced the LEGO-ECA (LEGO Error Correction Assembly Dataset), built upon 1,429 manuals. To create realistic failures, they injected Gaussian noise into existing assembly models to generate approximately 120,000 instances of misassembled parts.

Key results from testing on LEGO-ECA include:

  • Accuracy Boost: Component pose accuracy jumped from 38.58% to 49.31%.
  • Error Reduction: The Chamfer Distance (measuring 3D shape deviation) dropped from 28.98 to 12.61, a more than 50% improvement in structural fidelity.
  • Generalization: Even on datasets SCANet never saw during training, it improved assembly accuracy from 33.70% to 38.29%.

Key Insights

  • The Power of "Difference": Explicitly calculating the "delta" between the manual and the build makes it easier to spot bricks that are only slightly off-position.
  • Color Matters: Including 2D image data helps the model distinguish between identical geometries that differ only by color, preventing the model from incorrectly "correcting" one into the other's position.

Limitations and Future Work

While SCANet represents a significant step forward, it was conducted entirely in a simulated environment. Real-world factors like lighting and physical occlusions remain untested. Furthermore, the model currently focuses on errors made in the current step; recovering from errors buried under subsequent layers of bricks (multi-step error recovery) remains a vital challenge for future research.

Summary updated

BibTeX
@inproceedings{wan2024scanet,
  author = {Yuxuan Wan and Kaichen Zhou and Jinhong Chen and Hao Dong},
  title = {{SCANet: Correcting LEGO Assembly Errors with Self-Correct Assembly Network}},
  booktitle = {IEEE/RSJ International Conference on Intelligent Robots and Systems},
  year = {2024},
  doi = {10.1109/iros58592.2024.10801934},
}
Figure