Generating Physically Sound Training Data for Image Recognition of Additively Manufactured Parts

Tobias Nickchen, Stefan Heindorf, Gregor EngelsWACV 2021

TL;DR

This paper tackles the challenge of automatically recognizing additively manufactured parts, which is crucial for automating production lines. They propose generating training data for image recognition models using "physically sound" orientations of 3D models, rather than random ones, leading to significantly improved recognition accuracy, especially for complex parts.

What Problem Does This Paper Address?

Additive Manufacturing (AM), particularly Selective Laser Sintering (SLS), is increasingly used for industrial production. A key challenge in SLS is that multiple components are manufactured simultaneously in a "batch mode," nested together. After manufacturing, these components need to be identified and separated for subsequent processing. Currently, this identification is often done manually, which is inefficient for large production volumes. While computer vision systems can automate this, existing methods struggle with the vast number of possible component orientations and the need for data-driven approaches that can handle arbitrary parts. A major hurdle is training these recognition models, as real-world photos of manufactured parts are unavailable during the training phase. This necessitates using virtually generated images, but simply randomizing object orientations in these virtual images can lead to physically impossible scenarios, creating noisy training data and hindering recognition performance.

Methods

The core contribution of this paper is a novel approach to generating training data for 3D object recognition in the context of AM. Instead of using random orientations for virtual object rendering, they propose generating "physically sound" orientations. This involves several steps:

  1. Convex Hull Calculation: Determining the convex hull of the 3D object to identify potential contact surfaces with a horizontal plane.
  2. Stability Check: Using the object's Center of Mass (CoM) to filter out orientations that are physically unstable (i.e., the object would tip over).
  3. Height Criteria: Further refining orientations by excluding those where the CoM is excessively high relative to the lowest possible CoM, indicating low stability on a conveyor belt.

These physically sound orientations are then used to render virtual images from multiple viewpoints, which are fed into the RotationNet model. They also employ traditional image augmentation techniques (blurring, brightness/contrast changes, etc.) to further enhance robustness.

Datasets

The authors construct six virtual SLS build jobs using 3D models from the Thingi10K dataset. These build jobs vary in difficulty:

  • Random30, Random50, Random100: Datasets with 30, 50, and 100 randomly sampled objects, respectively.
  • Similar10, Similar30, Similar50: Datasets with 10, 30, and 50 manually selected objects from groups with high geometric similarity.

These datasets are made publicly available to ensure reproducibility. For evaluation, they also physically produced objects from the Random30 and Similar50 build jobs and captured real-world images.

Key Insights and Ideas

  • The "Physically Sound" Advantage: The most significant insight is that generating training data with orientations that are physically plausible dramatically improves recognition accuracy compared to random orientations. This is particularly true for geometrically similar objects.
  • Bridging the Simulation-to-Real Gap: By focusing on physically realistic orientations, the virtual training data better mimics real-world scenarios, leading to more effective transfer learning from simulated to real images.
  • Complexity Matters: The benefit of physically sound training data becomes more pronounced as the complexity of the object set increases (e.g., more geometrically similar objects).
  • Augmentation Synergy: Combining physically sound data generation with traditional image augmentation techniques yields the best results, further boosting accuracy and robustness.

Key Results

  • Significant Accuracy Improvement: On the difficult "Similar50" dataset, using physically sound training data with image augmentation achieved an accuracy of 84.21%, a substantial increase from 37.59% achieved with random training data and augmentation.
  • Near-Perfect Recognition: For simpler datasets like "Random30," physically sound training data consistently achieved over 98.51% accuracy, even without extensive augmentation.
  • Outperforming Random: Across all tested scenarios and datasets, physically sound training data consistently outperformed random training data, with the gap widening for more complex datasets.
  • Virtual to Real Transfer: The models trained on virtually generated physically sound data performed well on real-world images, demonstrating the effectiveness of the approach.

Limitations and Future Work

  • Prototype Limitations: The authors acknowledge that their physical image capture was done with a handheld camera without a fixed recognition station, leading to slight variations. They believe a fixed station would further improve results.
  • Rendering Realism: They suggest that aligning virtual rendering parameters (focal length, lighting, component positioning) more closely with the physical setup could enhance performance.
  • Future Work: The authors plan to compare their 2D multi-view approach with 3D-based methods (point clouds, voxels) in terms of practical feasibility and recognition rate. They also aim to explore applications beyond AM component recognition, such as industrial assembly lines for grasping, quality control, and sorting.

Summary updated

BibTeX
@inproceedings{nickchen2021generating,
  author = {Tobias Nickchen and Stefan Heindorf and Gregor Engels},
  title = {{Generating Physically Sound Training Data for Image Recognition of Additively Manufactured Parts}},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
  year = {2021},
}
Figure