TL;DR
Global Logic and Local Search (GLLS) is a training-free, dual-stream framework that enables Large Multimodal Models (LMMs) to perform highly accurate and verifiable industrial anomaly detection. By combining a tool-grounded structural logic check with an active, budget-constrained local search (via Monte Carlo Tree Search), GLLS eliminates the need for extensive defective training samples, making it ideal for early-stage industrial deployment.
What Problem Does This Paper Address?
Industrial anomaly detection is fundamentally different from general image recognition; it functions as a strict compliance audit against precise engineering standards rather than open-ended inference. When manufacturing lines are first deployed, defective samples are virtually non-existent ("cold-start" phase), meaning models must rely purely on text specifications and a handful of normal reference images.
While modern LMMs possess strong few-shot capabilities, they struggle with industrial inspection due to:
- Logic Hallucination and Visual Bias: LMMs frequently fail to ground abstract textual rules to pixel-level realities, sometimes misinterpreting manufacturer logos as defects or missing structural swaps entirely.
- Resolution Limits: Tiny defects (like a hairline scratch or a split zipper tooth) get lost in standard low-resolution model inputs.
Optimization-based methods try to fix this via fine-tuning, but they require large amounts of defective training data—precisely what is unavailable during early deployment.
Methods
GLLS introduces a training-free architecture that decouples evidence acquisition from final diagnostic reasoning. The system operates in two distinct phases:
1. Offline Phase: Part-Aware Visual-Logical Atlas (PVLA)
To build a structured knowledge base, GLLS uses an LLM (GPT-5) to expand basic dataset labels into detailed sub-component definitions and potential failure modes. Then, Segment Anything 3 (SAM 3) uses these text prompts to ground and extract visual primitives from a few normal reference images. This information is compiled into a heterogeneous directed graph mapping object classes to parts, normal anchors, and defect nodes.
2. Online Phase: Dual-Stream Verification
When a test image arrives, it is processed simultaneously by two parallel streams:
- Global & Logic Stream: SAM 3 acts as a deterministic fact extractor, measuring physical properties (counts, order, geometry) against the PVLA rules. A Global VLM reviews these structured outputs to catch macroscopic structural anomalies (e.g., missing components or swapped cables).
- Fine-Grained & Actions Stream: To catch minute defects without exhaustively scanning high-resolution images, the paper formulates local inspection as a budgeted Markov Decision Process (MDP). It uses Monte Carlo Tree Search (MCTS) guided by a lightweight anomaly heatmap to actively select and zoom in on the most informative local image crops.
Final Verdict
A final VLM agent fuses the structural report from the global stream, the high-resolution crops from the local search stream, and the retrieved context from the PVLA to deliver an auditable diagnostic decision.
Datasets
The authors evaluated GLLS on several prominent anomaly detection benchmarks:
- MMAD-QA: The primary benchmark used, focusing on closed-set visual question answering across MVTec-AD and VisA subsets.
- Cross-Dataset Generalization: To test robustness, the framework was evaluated on MPDD, DTD, and DAGM datasets.
Key Insights and Ideas
- Verification-First Grounding: Instead of asking an LMM to passively look at an image and guess if it's broken, GLLS forces deterministic vision tools to extract verifiable facts first. This converts a prone-to-hallucination VQA task into a structured compliance audit.
- Active Perception over Passive Scanning: Inspired by human cognitive vision theories, the model actively plans where to look closer using MCTS. This keeps the computational and token context budget fixed while maximizing the resolution on ambiguous regions.
- Graph-Structured In-Context Learning: Rather than relying on flat vector similarity searches for retrieval-augmented generation, navigating a hierarchical part-to-defect graph ensures the VLM receives highly relevant, component-scoped standards.
Key Results
- MMAD-QA Performance: GLLS achieved massive accuracy gains over base models. For example, pairing GLLS with Qwen3-VL (4B) in a 1-shot setting boosted Anomaly Discrimination accuracy on MVTec-AD from 70.5% to 96.4%.
- Cross-Dataset Robustness: In 1-shot binary anomaly detection across MPDD, DTD, and DAGM, GLLS achieved an average accuracy of 91.01%, outperforming proprietary models like GPT-4o and specialized CLIP-based detectors.
- Efficiency: The MCTS planning mechanism proved highly efficient, introducing a negligible search latency overhead of only 0.08s to 0.15s, making it highly practical for real-time inspection lines.
Limitations and Future Work
The authors noted that enforcing strict offline standards on passive, low-resolution inputs can occasionally increase false positives if the online verification stream isn't perfectly aligned.
For future work, the team aims to focus on automatically learning the local search policy using verifiable rewards and expanding the PVLA graph with richer contrastive edges to better differentiate highly similar defect types.
Summary updated