Generate Aligned Anomaly: Region-Guided Few-Shot Anomaly Image-Mask Pair Synthesis for Industrial Inspection

Yilin Lu, Jianghang Lin, Linhuang Xie, Kai Zhao, Yansong Qu, Shengchuan Zhang, Liujuan Cao, Rongrong Ji

Introduction

In industrial manufacturing, automated anomaly inspection—identifying defects such as cracks, stains, or missing components—is critical for quality control. However, training effective supervised models for this task is notoriously difficult due to the "data bottleneck." In optimized production environments, anomalies are rare occurrences, making it nearly impossible to collect the thousands of labeled examples typically required for deep learning.

Existing data augmentation strategies generally fall into two categories: model-free methods and generative methods. Model-free techniques, such as "cut-and-paste" augmentation, often produce unrealistic results that lack semantic logic (e.g., a scratch appearing in a vacuum rather than on a surface). Conversely, while generative models like GANs or Diffusion Models offer higher realism, they frequently struggle to produce accurate, pixel-perfect ground truth masks that align with the generated image. Furthermore, these models often fail to capture "logical" anomalies, such as the absence of a specific component in a complex assembly.

To address these challenges, we introduce Generate Aligned Anomaly (GAA), a framework that leverages the powerful priors of Latent Diffusion Models (LDMs) to synthesize high-fidelity anomaly image-mask pairs from only a few-shot examples.

Methodology

The GAA framework is built upon three primary pillars designed to decouple the appearance of an anomaly from its spatial context, ensuring both realism and alignment.

1. Localized Concept Decomposition

Standard generative models often attempt to learn an anomaly as a monolithic concept. GAA instead decomposes the embedding into two distinct components: a feature-focused embedding ($e_f$) and a position-focused embedding ($e_p$). The feature embedding captures the visual texture, color, and material properties of the defect, while the position embedding encodes the spatial constraints. This decoupling allows the model to learn "what" a crack looks like independently of "where" it is located, providing the flexibility to synthesize defects across various valid regions of a normal image.

2. Adaptive Multi-Round Anomaly Clustering

Industrial datasets often use broad, heterogeneous labels like "defect." This semantic diversity can confuse generative models, especially in few-shot settings. GAA employs an automated multi-round KMeans clustering approach, guided by the Davies-Bouldin index, to refine these broad categories into "semantically pure" sub-clusters. For instance, it might separate "missing bristles" from "oil stains" on a toothbrush. By training on these refined sub-categories, the model achieves much higher generative clarity.

3. Region-Guided Mask Synthesis and Filtering

To ensure that synthesized anomalies appear in logical locations, GAA utilizes a "Region-Aware Mask Propagation" strategy. By integrating foundation models like the Segment Anything Model (SAM) and SegGPT, the framework identifies valid semantic regions (e.g., the specific area where a nut should be in a snack box) and adaptively fits synthetic masks into those regions.

To maintain high data quality, GAA introduces the Anomaly Region Score (ARS). This metric evaluates the alignment between the generated anomaly and its intended mask. Samples that exhibit "hallucinations" or poor alignment are automatically filtered out, ensuring that only high-quality image-mask pairs are used for downstream training.

Experimental Evaluation

The researchers evaluated the GAA framework on two primary industrial benchmarks:

  • MVTec AD: The industry standard for structural anomalies, featuring 15 categories of objects and textures with defects like scratches and cracks.
  • MVTec LOCO: A more complex dataset focused on logical anomalies, such as misplaced components or incorrect quantities of items.

Key Results

GAA establishes a new state-of-the-art for anomaly synthesis and downstream task performance:

  • Localization Performance: When GAA-generated data was used to train a standard U-Net, it achieved 96.3% AUROC and 76.8% Average Precision (AP) on MVTec AD, significantly outperforming previous generative methods like AnoDiff.
  • Classification Accuracy: On the task of identifying specific anomaly types, GAA reached 84.7% accuracy, representing an 18.6% improvement over the previous state-of-the-art.
  • Qualitative Realism: Visual analysis confirms that GAA-generated anomalies integrate seamlessly into the base objects, avoiding the jagged edges and "floating" artifacts that plague simpler augmentation methods.

Conclusion and Future Work

The Generate Aligned Anomaly (GAA) framework demonstrates that by decoupling appearance from location and utilizing region-guided synthesis, it is possible to generate high-quality training data for industrial inspection from minimal samples. While GAA excels at texture and local component modifications, it currently faces limitations regarding complex global layout changes—such as completely relocating a large object within a scene while maintaining global structural logic. Future research will focus on enhancing the model's understanding of these high-level spatial relationships to further broaden the scope of synthetic industrial data.

Summary updated

BibTeX
@inproceedings{lu2025generate,
  author = {Yilin Lu and Jianghang Lin and Linhuang Xie and Kai Zhao and Yansong Qu and Shengchuan Zhang and Liujuan Cao and Rongrong Ji},
  title = {{Generate Aligned Anomaly: Region-Guided Few-Shot Anomaly Image-Mask Pair Synthesis for Industrial Inspection}},
  booktitle = {ACM Multimedia},
  year = {2025},
  doi = {10.1145/3746027.3754920},
}
Figure