Global Logic and Local Search: Dual-Stream Multimodal In-Context Learning for Verifiable Industrial Anomaly Detection

Runzhi Deng, Yundi Hu, Yiming Zhong, Zhao Wang, Xixi Liu, Hongsong Wang, Caifeng Shan, Fang ZhaoarXiv 2026 · arXiv

TL;DR

Global Logic and Local Search (GLLS) is a training-free, dual-stream framework that enables Large Multimodal Models (LMMs) to perform highly accurate and verifiable industrial anomaly detection. By combining a tool-grounded structural logic check with an active, budget-constrained local search (via Monte Carlo Tree Search), GLLS eliminates the need for extensive defective training samples, making it ideal for early-stage industrial deployment.

What Problem Does This Paper Address?

Industrial anomaly detection is fundamentally different from general image recognition; it functions as a strict compliance audit against precise engineering standards rather than open-ended inference. When manufacturing lines are first deployed, defective samples are virtually non-existent ("cold-start" phase), meaning models must rely purely on text specifications and a handful of normal reference images.

While modern LMMs possess strong few-shot capabilities, they struggle with industrial inspection due to:

  1. Logic Hallucination and Visual Bias: LMMs frequently fail to ground abstract textual rules to pixel-level realities, sometimes misinterpreting manufacturer logos as defects or missing structural swaps entirely.
  2. Resolution Limits: Tiny defects (like a hairline scratch or a split zipper tooth) get lost in standard low-resolution model inputs.

Optimization-based methods try to fix this via fine-tuning, but they require large amounts of defective training data—precisely what is unavailable during early deployment.

Fig. 1: Motivation and Performance Overview. The left panels visualize critical failure modes in industrial inspection, where LMMs typically suffer from Logic Hallu- cination and Visual Bias (top), while specialized models of smaller scale struggle with Semantic Blindness and Resolution Limits (bottom). We address these limitations by incorporating a tool-grounded Global Logic stream and an active Local Search stream to support auditable inspection. The right panels present quantitative analyses showing that the GLLS framework improves over representative general-purpose baselines under a controlled budget of computation across five metrics on the MMAD-QA benchmark.

Methods

GLLS introduces a training-free architecture that decouples evidence acquisition from final diagnostic reasoning. The system operates in two distinct phases:

1. Offline Phase: Part-Aware Visual-Logical Atlas (PVLA)

To build a structured knowledge base, GLLS uses an LLM (GPT-5) to expand basic dataset labels into detailed sub-component definitions and potential failure modes. Then, Segment Anything 3 (SAM 3) uses these text prompts to ground and extract visual primitives from a few normal reference images. This information is compiled into a heterogeneous directed graph mapping object classes to parts, normal anchors, and defect nodes.

2. Online Phase: Dual-Stream Verification

When a test image arrives, it is processed simultaneously by two parallel streams:

  • Global & Logic Stream: SAM 3 acts as a deterministic fact extractor, measuring physical properties (counts, order, geometry) against the PVLA rules. A Global VLM reviews these structured outputs to catch macroscopic structural anomalies (e.g., missing components or swapped cables).
  • Fine-Grained & Actions Stream: To catch minute defects without exhaustively scanning high-resolution images, the paper formulates local inspection as a budgeted Markov Decision Process (MDP). It uses Monte Carlo Tree Search (MCTS) guided by a lightweight anomaly heatmap to actively select and zoom in on the most informative local image crops.

Final Verdict

A final VLM agent fuses the structural report from the global stream, the high-resolution crops from the local search stream, and the retrieved context from the PVLA to deliver an auditable diagnostic decision.

Fig. 2: System Overview. Offline Construction: The Part-Aware Visual-Logical Atlas (PVLA) is constructed from natural language specifications and a few normal reference images. The SAM 3 segmentation model extracts visual primitives to populate the atlas with canonical visual norms and executable logic. Online Dual-Stream Inference: (1) The Global & Logic Stream executes tool-grounded checks via the SAM 3 model to validate structural integrity. (2) In parallel, the Fine-Grained & Actions Stream formulates the inspection process as a budgeted Markov Decision Process (MDP), utilizing MCTS to actively zoom in on ambiguous regions. (3) A Vision-Language Model (VLM) performs the final Reference-Guided In-Context Verification by fusing structural verdicts and local evidence chains.

Datasets

The authors evaluated GLLS on several prominent anomaly detection benchmarks:

  • MMAD-QA: The primary benchmark used, focusing on closed-set visual question answering across MVTec-AD and VisA subsets.
  • Cross-Dataset Generalization: To test robustness, the framework was evaluated on MPDD, DTD, and DAGM datasets.

Key Insights and Ideas

  • Verification-First Grounding: Instead of asking an LMM to passively look at an image and guess if it's broken, GLLS forces deterministic vision tools to extract verifiable facts first. This converts a prone-to-hallucination VQA task into a structured compliance audit.
  • Active Perception over Passive Scanning: Inspired by human cognitive vision theories, the model actively plans where to look closer using MCTS. This keeps the computational and token context budget fixed while maximizing the resolution on ambiguous regions.
  • Graph-Structured In-Context Learning: Rather than relying on flat vector similarity searches for retrieval-augmented generation, navigating a hierarchical part-to-defect graph ensures the VLM receives highly relevant, component-scoped standards.

Key Results

  • MMAD-QA Performance: GLLS achieved massive accuracy gains over base models. For example, pairing GLLS with Qwen3-VL (4B) in a 1-shot setting boosted Anomaly Discrimination accuracy on MVTec-AD from 70.5% to 96.4%.
  • Cross-Dataset Robustness: In 1-shot binary anomaly detection across MPDD, DTD, and DAGM, GLLS achieved an average accuracy of 91.01%, outperforming proprietary models like GPT-4o and specialized CLIP-based detectors.
  • Efficiency: The MCTS planning mechanism proved highly efficient, introducing a negligible search latency overhead of only 0.08s to 0.15s, making it highly practical for real-time inspection lines.

Limitations and Future Work

The authors noted that enforcing strict offline standards on passive, low-resolution inputs can occasionally increase false positives if the online verification stream isn't perfectly aligned.

For future work, the team aims to focus on automatically learning the local search policy using verifiable rewards and expanding the PVLA graph with richer contrastive edges to better differentiate highly similar defect types.

Summary updated

Abstract

Large Multimodal Models (LMMs) show strong few-shot generalization, but industrial anomaly detection remains difficult because defects are small, input resolution is limited, and textual standards are not always grounded in visual evidence. Recent optimization-based methods improve alignment through fine-tuning, but they often require many defective samples, which are unavailable in early deployment. We present Global Logic and Local Search (GLLS), a training-free framework for reference-guided multimodal in-context verification. GLLS uses a Part-Aware Visual-Logical Atlas to organize normal references and structured specifications in the inference context. It combines a Global & Logic Stream, where SAM 3 extracts partially checkable visual facts, with a Fine-Grained & Actions Stream, where MCTS selects local evidence crops under a fixed budget. Experiments on MMAD-QA and additional anomaly detection datasets show consistent gains over matched and general-purpose baselines, while keeping the final diagnostic decision traceable to explicit visual evidence throughout the inspection trace.
BibTeX
@misc{deng2026global,
  author = {Runzhi Deng and Yundi Hu and Yiming Zhong and Zhao Wang and Xixi Liu and Hongsong Wang and Caifeng Shan and Fang Zhao},
  title = {{Global Logic and Local Search: Dual-Stream Multimodal In-Context Learning for Verifiable Industrial Anomaly Detection}},
  year = {2026},
  eprint = {2607.03817},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2607.03817},
}
Figure