A surface defect such as a hairline scratch or a shallow dent can be plainly visible under one lighting angle and invisible under the next. Under a third angle, a harmless specular highlight can look exactly like a defect. Visual Anomaly Detection under Complex View-Illumination Interplay built a benchmark, M²AD, that photographs each object under 120 combinations of camera viewpoint and light direction. On it, a strong anomaly detector (Dinomaly) drops from 99.6% AUROC on the standard MVTec AD benchmark to 81.3%. The same paper reports a "fusion paradox": averaging anomaly scores across many lighting conditions makes results worse, because the lights that hide the defect contribute noise. We propose to stop treating lighting as a nuisance to average away. Instead, the inspection system should treat each lighting condition as a question it can choose to ask. It looks once, forms a draft hypothesis about where a defect might be, and then picks the next light whose answer best separates "real defect" from "lighting artifact." It stops once the evidence is decisive.
The decision rule comes from rare-event physics. There, a faint candidate signal is accepted only when independent channels agree in a way that background cannot mimic. Enhancing the significance of astrophysical events with multimessenger coincidences raises the probability that a weak gravitational-wave trigger is real from 7.7% to 86% by checking whether a neutrino arrived at the right place and time. The equivalent of "background" in inspection is glare, shading and texture. These artifacts follow the light direction in predictable, learnable patterns. A real defect is fixed to the object surface. So the system will learn, from defect-free parts only, which lighting conditions tend to produce false alarms together: a co-occurrence model of the background. It then queries lights where background and defect predict different outcomes.
Core Claim
Under a fixed budget of image acquisitions per part, we expect an adaptive loop to beat both of the simpler options. The first is a single good view. The second is any fixed subset of lights chosen in advance, however well it is chosen. The loop has three parts: a draft anomaly map, a learned model of which lighting conditions share false alarms, and a Bayesian coincidence update. We further claim the gain comes specifically from the background co-occurrence model, not from merely seeing more images. The new element is the closed loop. M²AD evaluates fixed acquisition sets, and the physics coincidence framework is applied offline after detection. Neither chooses its next observation from its current uncertainty.
Research Program
The loop borrows its structure from retrieval-augmented question answering. Retrieving, Rethinking and Revising (CoV-RAG) shows that revising a search query after a draft answer works better than optimizing the query beforehand, because the draft exposes the specific gap. Chain-of-Rewrite uses the model's own uncertainty to down-rank distractor documents and recycle candidate answers into a second query. Our analogue works as follows:
- Draft. A first image under a default light produces an anomaly map from a frozen off-the-shelf detector (Dinomaly).
- Candidates. The few highest-scoring regions become the hypotheses.
- Query. For each candidate, the system chooses the next lighting condition to acquire.
- Retrieval. The image under that light is the "retrieved document."
These systems also warn of a failure mode we must guard against. Unbounded loops amplify distractors, so our loop has an explicit stopping rule and a hard budget.
The background model is fit to normal training parts only. For each viewpoint, images taken under different lights are, we assume, pixel-aligned; checking this on M²AD is our first step. We run the frozen detector on every lighting condition of every defect-free part and record, per location, the vector of anomaly scores across the 10 lights. From these vectors we estimate a conditional density $p(s_j \mid s_S, \text{normal})$. This is the score under light $j$, given scores already observed under the set of lights $S$. It is fit as a per-location Gaussian copula over lights, with pointwise-mutual-information statistics between lights as a diagnostic.
The density captures the structure of background. For example, a highlight under light 3 implies a highlight under neighboring light 4 but not under the opposite light 8. This mirrors how Exploring the Bounds of the Utility of Context for Object Detection finds that co-occurrence context is good at separating real objects from clutter. The same paper also constrains our design. It finds that context barely helps correct where a box is (about 55% accuracy, near chance). We therefore use co-occurrence only to decide whether a fixed, pixel-aligned location is a defect. Localization still comes from the detector's per-pixel map.
The defect model must be deliberately weak, because unsupervised inspection has no defect labels at training time. We posit that a real defect yields elevated scores at a fixed surface location under some unknown subset of lights. Each light reveals it independently with probability $q$, and $q$ is estimated from synthetic cut-and-paste defects on normal parts. This dependence on an assumed signal model is exactly the weakness Veske et al. acknowledge for emission models, so we report sensitivity to $q$ rather than hide it. For a candidate location, the log-odds that it is a defect after observing lights $S$ is updated as
$$\log \frac{P(D \mid s_S)}{P(N \mid s_S)} = \log \frac{\pi}{1-\pi} + \sum_{j \in S} \log \frac{p(s_j \mid s_{S_{<j}}, D)}{p(s_j \mid s_{S_{<j}}, N)}.$$
Here $D$ and $N$ denote defect and normal, $\pi$ is the prior defect rate, and $S_{<j}$ is the set of lights acquired before $j$. The next light is the one with the largest expected change in this log-odds, summed over the current candidates. The loop stops when every candidate's posterior leaves the interval $[\tau, 1-\tau]$ or the budget $k$ is spent.
The analogy to particle physics is concrete. Topological background discrimination in PandaX-III rejects background by about seven orders of magnitude because signal and background have different shapes across several features. Our "shape" is the pattern of scores across lights. Our loop differs from PandaX-III in one respect: it chooses which features to measure.
Baselines
The naive reference is a single image under the default light, scored by Dinomaly on the M²AD-Invariant protocol. For this setting the cited benchmark reports 81.3% AUROC (M²AD). We will reproduce this number before anything else.
The strongest existing method is all-condition fusion under the M²AD-Synergy protocol, run with the same frozen detector. We will run it in three variants: plain averaging (the paradox setting), max pooling, and a small learned attention fusion trained on normal parts. We found no reported result on Lacuna for Synergy-protocol fusion AUROC, so we will produce these numbers ourselves under identical preprocessing and resolution. All-condition fusion uses up to 120 acquisitions. Beating it means matching its AUROC with $k \le 10$ acquisitions, or exceeding it outright.
The most dangerous competitor is the best static subset: the $k$ lights per viewpoint chosen greedily on the validation split to maximize AUROC, then fixed for every part. To keep the comparison fair, it gets the same validation data, the same frozen detector, and the same synthetic-defect validation set we use to tune $\tau$ and $q$. It also gets an equal search budget over subsets. Beating it means a higher AUROC at every $k$ from 2 to 6, with confidence intervals from part-level bootstrap.
Three ablations isolate the key ingredient. First, adaptive selection with the co-occurrence model replaced by an independence assumption across lights. Second, adaptive selection by detector uncertainty alone, with no background model at all. Third, random adaptive selection with the same stopping rule. The proposal's claim only holds if the full model beats the first two ablations.
Why not just…?
Why not just pick the best few lights once, offline? This is the real threat. If whether a defect is visible depends mainly on material and viewpoint, a fixed subset captures most of the benefit, and adaptivity buys nothing. Adaptivity only pays when the revealing light depends on where the defect sits on each individual part. The static-subset baseline above settles this directly. We will also report how often the adaptive policy's chosen lights differ from the static subset, per material. If the gap vanishes on most materials, we will report that the problem is non-adaptive.
Why not just fuse all 120 images with a learned weighting? If acquisition were free, a learned fusion might match us. M²AD itself shows naive averaging fails, and a learned weighting is the natural fix. We include that weighting as a baseline. Our claim is about accuracy per acquisition: on a production line, each re-lighting costs cycle time. We will report the full AUROC-versus-acquisitions curve so a reader can see where all-view fusion wins.
Why not just train the detector to be lighting-invariant, with augmentation or disentanglement? Generalization Gap in Data Augmentation finds that synthetic lighting augmentation leaves a gap to real multi-light training (95.1% versus 99.7% on ResNet). Augmentation cannot recreate light interacting with surface geometry. More fundamentally, a defect that is physically invisible under a given light cannot be recovered by any invariance. Rare Event Detection Using Disentangled Representation Learning separates lighting from content using pairs of images without events. We will train its dual-branch model on normal pairs taken under different lights as a single-image baseline. It still cannot ask for a better image, which is the capability we are testing.
Why not just enhance or denoise the image first? A Comprehensive Study of Object Tracking in Low-Light Environments found that enhancement and denoising leave failures caused by look-alike distractors intact. A highlight that looks like a scratch is a confusion of meaning, not a noise problem. We treat preprocessing as compatible with our loop rather than as a competitor, and will test one denoised variant.
Proposed Experiments
All experiments use M²AD. We train on normal parts and evaluate image-level and pixel-level AUROC under both protocols, with acquisition counts from 1 to 10 per viewpoint, broken down by material.
The first decisive check is the curve of AUROC against the number of acquisitions. For the claim to hold, the full method should lie above the best static subset and above both co-occurrence ablations at small $k$. It should also approach or exceed all-condition fusion by roughly $k=6$.
The second check is mechanistic. On parts where a highlight triggers a false alarm, we will show that the selected next light is one the background model predicts will not reproduce the highlight. We will then measure how often the false alarm's posterior collapses afterwards. This is our version of the consensus gating in Learning a Robust Society of Tracking Parts, which updates its model only on frames where independent pathways agree.
The third check is sensitivity to the signal model. We will sweep $q$ and the prior $\pi$ and report how much AUROC moves. A result that holds only for a narrow $q$ would mean the gain comes from tuning, not from the coincidence structure.
Finally, we will report wall-clock time per part including simulated acquisition. Otherwise the latency concerns raised for iterative retrieval loops would apply here unexamined.
References
- Visual Anomaly Detection under Complex View-Illumination Interplay: A Large-Scale Benchmark
- Enhancing the significance of astrophysical events with multimessenger coincidences
- Retrieving, Rethinking and Revising: The Chain-of-Verification Can Improve Retrieval Augmented Generation
- Chain-of-Rewrite: Aligning Question and Documents for Open-Domain Question Answering
- Exploring the Bounds of the Utility of Context for Object Detection
- Topological background discrimination in the PandaX-III neutrinoless double beta decay experiment
- Generalization Gap in Data Augmentation: Insights from Illumination
- Rare Event Detection Using Disentangled Representation Learning
- A Comprehensive Study of Object Tracking in Low-Light Environments
- Learning a Robust Society of Tracking Parts Using Co-occurrence Constraints