TL;DR
ADOPD introduces a reference-privileged on-policy distillation framework that bakes the benefits of image-comparison directly into an MLLM's parameters during training. By evaluating student rollouts against both matched and mismatched reference images, it trains a query-only model to achieve superior zero-shot anomaly detection without the computational overhead of inference-time reference retrieval.
What Problem Does This Paper Address?
Industrial Anomaly Detection (IAD) requires identifying fine-grained defects or irregularities in complex visual data. While Multimodal Large Language Models (MLLMs) excel at open-set reasoning, their open-domain representations struggle to capture subtle, product-specific anomaly patterns.
Providing a "normal" reference image at inference time (one-shot inference) significantly helps MLLMs by giving them a visual baseline for comparison. However, retrieving and processing a reference image for every single query during deployment introduces substantial computational overhead and latency. The authors address a compelling question: Can the benefits of reference comparison be internalized into the model's parameters during training, allowing the model to maintain high accuracy during zero-shot inference?
Methods
The authors frame this as a "privileged information" problem, where a strong, reference-aware teacher model supervises a query-only student model during training. However, standard on-policy distillation (OPD) can fail if the teacher relies on language priors or query-image cues rather than actual reference comparison.
To solve this, they propose ADOPD (Reference-Privileged On-Policy Distillation), which separates what the model should learn from how strongly it should learn it:
- On-Policy Sampling: The trainable student model samples text trajectories (rollouts) based solely on the query image and inspection question.
- Asymmetric Reference Intervention: A frozen teacher model replays and evaluates these exact same student-generated tokens under two distinct reference views:
- Matched Reference ($r_{mat}$): A regional crop that preserves the valid query-reference relation.
- Mismatched Reference ($r_{mis}$): A regional crop from an opposite anomaly state that deliberately breaks the relation.
- Token-Level Direction: The teacher-to-student log-probability ratio under the matched reference defines the token-level advantage, guiding the student on what to learn.
- Sequence-Level Calibration: The teacher's likelihood gap between the matched and mismatched views serves as a proxy for how much the teacher actually relied on the reference. This gap calibrates a group-relative sequence weight, determining how strongly the rollout should update the student's parameters.
Datasets
- Training: The authors constructed 6,000 paired training records from Real-IAD, covering 30 product categories and 5 camera views. Each defective image was paired with a normal counterfactual image to generate the matched and mismatched regional crops for the teacher.
- Evaluation: Models were evaluated on the comprehensive MMAD benchmark, which aggregates four major industrial anomaly detection datasets: MVTec-AD, VisA, MVTec-LOCO, and GoodsAD.
Key Insights and Ideas
- The "Control Group" for Teachers: MLLM teachers can easily hallucinate or rely on language shortcuts. By introducing a mismatched reference as a control, ADOPD effectively audits the teacher: if the teacher gives a response high praise regardless of whether the reference is right or wrong, the framework discounts that feedback.
- Fixing Exposure Bias: Unlike standard Supervised Fine-Tuning (SFT)—which is an off-policy method training models on fixed teacher targets—ADOPD's on-policy nature ensures the student model is evaluated on its own generation trajectories, leading to better generalization at inference time.
- Granularity Matters: The authors discovered that supplying full-image references to the teacher was less effective than supplying localized regional crops. Constraining the teacher's privileged view to anomaly-relevant regions produces a much cleaner distillation signal.
Key Results
- State-of-the-Art Zero-Shot Performance: ADOPD achieved an average accuracy of 77.31% on the MMAD benchmark under zero-shot inference, outperforming all open-source and domain-specific baselines.
- Beating the One-Shot Baseline: ADOPD improved its Qwen3-VL-4B backbone by 6.14 percentage points and actually outperformed the backbone's one-shot setting by 2.64 points, despite not having access to any reference images at inference time.
- Targeted Improvements: The performance gains were heavily concentrated in comparison-sensitive tasks like anomaly discrimination (+12.72 points) and defect localization (+2.49 points), proving that the model successfully internalized a fine-grained inspection strategy.
Limitations and Future Work
The primary limitation of ADOPD is its heavy reliance on carefully paired training images and precise defect annotations to construct the matched and mismatched regional crops for the teacher. The authors note that future work will explore moving away from these strict requirements by utilizing weaker or automatically generated privileged references to scale up training data more efficiently.
Summary updated