AgentIAD: Tool-Augmented Single-Agent for Industrial Anomaly Detection

Junwen Miao, Penghui Du, Yi Liu, Yu Wang, Yan WangarXiv 2025 · DOI · arXiv

Abstract

Industrial Anomaly Detection (IAD) requires identifying subtle, localized defects that are often missed by single-pass Vision-Language Models (VLMs). We introduce AgentIAD, a single-agent framework that treats inspection as an iterative, tool-driven conversation. By utilizing a "Perceptive Zoomer" for fine-grained analysis and a "Comparative Retriever" for reference-based validation, AgentIAD achieves a state-of-the-art accuracy of 97.62% on the MMAD benchmark. Our approach demonstrates that lightweight models, when equipped with agentic reasoning and specialized tools, can outperform much larger proprietary models like GPT-4o.

1. Introduction

Industrial inspection is a "needle in a haystack" problem where defects—such as microscopic scratches or missing components—are rare and highly localized. Traditional AI approaches often rely on "black-box" systems that lack interpretability, while modern VLMs typically attempt to analyze images in a single pass. These single-pass models frequently overlook fine details due to resolution constraints and lack a mechanism to compare suspicious regions against "normal" reference samples. AgentIAD addresses these limitations by transforming the inspection process into a multi-stage, agentic workflow.

2. The AgentIAD Framework

AgentIAD equips a lightweight VLM with two specialized visual tools to mimic human inspection behavior:

  1. Perceptive Zoomer (PZ): This tool enables the agent to crop and magnify specific regions of interest (ROI), allowing for the detection of fine-grained defects that are invisible at standard resolutions.
  2. Comparative Retriever (CR): This tool allows the agent to query a "normal" exemplar image from the same category. This is crucial for identifying "logical" anomalies, such as a missing item in a set, where the defect is only apparent when compared to a canonical reference.
Comparison of standard VLM reasoning vs. AgentIAD's tool-augmented approach.

As shown in the figure above, standard VLMs often fail to detect subtle discoloration or structural imbalances. In contrast, AgentIAD uses its tools to zoom in on suspicious areas and verify them against reference knowledge, leading to accurate identification of defects like "disproportionate arrangement."

3. Training and Optimization

To ensure effective tool utilization, AgentIAD undergoes a two-stage training pipeline:

  • Perceptive Supervised Fine-Tuning (SFT): The model learns the "grammar" of inspection—describing scenes, invoking tools, and interpreting results—using reasoning trajectories generated via GPT-4o.
  • Agentic Reinforcement Learning (RL): Using Group Relative Policy Optimization (GRPO), the model is refined through a dual-reward system. A Perception Reward ensures spatial alignment and classification accuracy, while a Behavior Reward penalizes redundant tool use, encouraging efficiency.

4. Experimental Results

AgentIAD was evaluated across major benchmarks, including MVTec, VisA, MVTec-LOCO, and GoodsAD. Key findings include:

  • State-of-the-Art Performance: AgentIAD achieved an average classification accuracy of 97.62% on the MMAD benchmark.
  • Outperforming Proprietary Models: Despite having only 3 billion parameters (Qwen2.5-VL-3B), AgentIAD significantly outperformed GPT-4o (68.63%) and GPT-4.1 (71.60%).
  • Interpretable Reasoning: The model provides transparent Chain-of-Thought (CoT) traces, explaining its decision-making process (e.g., "I see a dark spot... let me zoom in... comparing this to a normal sample...").

5. Conclusion

AgentIAD demonstrates that the reasoning process is often more critical than raw parameter count in specialized tasks. By providing VLMs with the ability to "zoom and compare," we achieve higher accuracy and greater transparency in industrial inspection. Future work will explore expanding the toolbox to include 3D point cloud analysis and multi-spectral imaging to address even more complex material defects.

Summary updated

BibTeX
@misc{miao2025agentiad,
  author = {Junwen Miao and Penghui Du and Yi Liu and Yu Wang and Yan Wang},
  title = {{AgentIAD: Tool-Augmented Single-Agent for Industrial Anomaly Detection}},
  year = {2025},
  eprint = {2512.13671},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2512.13671},
}
Figure