Training a Computer Vision Model for Commercial Bakeries with Primarily Synthetic Images

Thomas H. Schmitt, Maximilian Bundscherer, Tobias Bocklet

Abstract

Commercial bakeries face significant labor bottlenecks in tracking returned unsold goods. This paper presents an automated object detection approach using state-of-the-art models trained primarily on synthetic data. By leveraging a "Copy-Paste" augmentation pipeline and generative AI (pix2pix), we bridge the data gap inherent in niche industrial applications. Our best-performing model, YOLOv8x, achieved a 90.3% [email protected] on a real-world test set of 25 baked good categories, demonstrating the viability of synthetic-heavy training for small-to-medium enterprises (SMEs).

Introduction

In the industrial food sector, resource efficiency is paramount. Bakeries often reprocess returned products into animal feed or breadcrumbs, but manual tracking of these returns is labor-intensive. SMEs often struggle with the "data gap"—the lack of large, annotated datasets required to train robust computer vision models for their specific product lines. This study explores a pipeline to generate high-quality synthetic training data from a limited set of controlled images.

Methodology

The researchers developed a multi-stage pipeline to maximize utility from minimal real-world data:

  1. Semi-Automatic Annotation: The Segment Anything Model (SAM) was utilized to generate precise segmentation masks for individual baked goods, replacing manual bounding box drawing.
  2. Copy-Paste Augmentation: Segmented items were "pasted" onto various backgrounds to simulate crowded, realistic return scenarios. An "object scale check" was implemented to maintain realistic relative sizes between different pastry types.
  3. Generative Modeling: To increase intra-class variance, the team evaluated GAN-based approaches. While CycleGAN suffered from training collapse on this specialized dataset, pix2pix (a paired image-to-image translation model) successfully reproduced the visual nuances of baked goods.
  4. Object Detection: The latest YOLOv8 and YOLOv9 architectures (specifically the 'x' and 'e' variants) were fine-tuned and compared.
Comparison of a baseline training image (left) and a complex synthetic scene generated via Copy-Paste augmentation (right).

Datasets

The study utilized five distinct data subsets to build a robust model:

  • Baseline ($train_b$): 2,050 images of individual items on a controlled drying tray.
  • Complementary ($train_c$): 273 images of "unknown" items to improve robustness.
  • Auxiliary ($train_d$): Empty background images to minimize false positives.
  • Synthetic ($train_s$): 2,042 images generated via pix2pix.
  • Test Set: 109 real-world images from an alpha test environment, featuring high density (avg. 21 items/image), occlusions, and varied lighting.
Relative distribution of the 25 baked good categories across the training and test datasets.

Key Insights

  • Generative Success: The failure of CycleGAN versus the success of pix2pix highlights that for small, niche datasets, paired translation is more stable.
  • Preprocessing for GANs: Removing backgrounds from training images before feeding them into the GAN significantly reduced distortions in the synthetic output.
  • Architecture Comparison: Performance differences between YOLOv8x and YOLOv9e were marginal, suggesting that for industrial fine-tuning, data quality outweighs minor architectural iterations.
  • Color Importance: Grayscale training degraded performance, confirming that color is a critical feature for distinguishing between various types of browned pastries.

Results

The models demonstrated high accuracy on a test set significantly more complex than the training data. The YOLOv8x model, trained on the combined real and synthetic dataset, achieved the peak [email protected] of 90.3% and a maximum F1-score of 0.82. The YOLOv9e baseline followed closely at 90.1%. These results prove that models can be trained primarily on synthetic images with minimal performance loss in niche industrial settings.

Conclusion

This research demonstrates a viable path for SMEs to adopt AI-driven automation despite data scarcity. Future work will focus on improving the spatial logic of the Copy-Paste pipeline to better simulate natural stacking and exploring more advanced generative models like Stable Diffusion for increased training diversity.

Summary updated

BibTeX
@inproceedings{schmitt2024training,
  author = {Thomas H. Schmitt and Maximilian Bundscherer and Tobias Bocklet},
  title = {{Training a Computer Vision Model for Commercial Bakeries with Primarily Synthetic Images}},
  booktitle = {INFORMATIK},
  year = {2024},
  doi = {10.18420/inf2024\_150},
}
Figure