Reconstruction-based anomaly detection works like this: train a generator only on defect-free images, ask it to redraw a test image, and treat the parts it refuses to redraw as defects. The approach nearly saturates tidy factory benchmarks. It breaks, though, the moment the part is rotated, the light changes, or the surface is naturally rough. The redrawn image then differs from the input everywhere, not just at the defect. The generator has no way to know what pose, lighting, or texture the "healthy version of this particular image" should have.
We propose to give it that information directly. The generator receives the test image plus one or a few defect-free reference images of the same kind of part, taken under arbitrary conditions. It is trained to output a defect-free version of the test image in the test image's own pose and lighting, borrowing what "normal" looks like from the references. The difference between input and output then contains the defect and very little else. We test this on the real-world robustness benchmarks where current reconstructors either collapse or have never been evaluated.
Core Claim
The dominant failure of reconstruction detectors under environmental change is a mismatch of frames. The residual $|x - \hat{x}|$, where $x$ is the test image and $\hat{x}$ its reconstruction, compares pixels that no longer correspond once pose, illumination, or texture statistics drift away from the training set. Our claim is that the generator's normal prior should be conditioned on references rather than fixed at training time. A reconstructor that transfers normal content from a reference into the query's own geometry and lighting should keep reconstruction's strength in pixel-level localization while recovering image-level detection under physical shifts.
The motivating evidence is spread across the file. On the unaligned production-line dataset ReinAD, the best baseline falls to 69.0% image AUROC, against over 90% on MVTec AD. Robust AD finds that pose changes are the most destructive shift, and that robustness to synthetic noise does not predict robustness to physical change. On road tunnel walls, the engineering-structures evaluation shows the conditioned diffusion model DDAD dropping to 71.89% pixel AUROC because natural concrete variation reads as reconstruction failure.
How This Differs From the Closest Work
Four lines of work sit close to this idea, and each leaves the specific combination open.
Two lines use references but do not generate. ReinAD already compares a query to normal reference prototypes. It does this through heavy 4D correlation matching and deliberately avoids generation. Its output is therefore a matching score, not a healed image in the query's frame. The 3D Gaussian Splatting pipelines benchmarked on RAD also use a reference: they render a normal view at the estimated pose and subtract it. They reach 0.984 pixel AUROC (PIAD) but only 0.524 to 0.634 image AUROC, because rendering artifacts create spurious residuals. This is the result our proposal must beat. We replace explicit rendering and exact registration with a learned conditional generator, which we hypothesize can absorb small misregistrations and view-dependent specular effects instead of reporting them.
Two lines address nuisance without references. The Augmented Autoencoder of Sundermeyer et al. reconstructs a clean canonical view from cluttered, relit inputs. This is the opposite choice from ours: it moves the image to a canonical frame for pose retrieval, which would make pixel residuals meaningless for defects. FiCo separates style from invariant content, but only inside a student-teacher feature pipeline. It was tested on digital corruptions and on the PACS style-shift benchmark, not on geometric shifts.
On the method side, we borrow three components. From DiffusionAD we take the one-step denoising design, for latency. From DRÆM we take synthetic defect pasting and a learned residual head. From DDAD we take conditioning of the reverse diffusion trajectory on the input.
Research Program
The model is a diffusion denoiser, meaning a generator trained to remove added noise from an image. Its inputs are a noised query image and features from $k$ reference images, with $k$ between 1 and 4. The reference features enter through cross-attention, so the network can look up matching regions in the references without explicit alignment.
Training needs only normal images and is self-supervised. We draw two or more normal captures of the same category taken under different conditions. Examples are different RAD viewpoints, different RobustAD pose and lighting settings, or neighbouring sane tunnel crops. One capture becomes the query. We paste synthetic defects onto it using DRÆM-style Perlin masks, with an added step that matches the brightness of the pasted texture to the query's local shading. The training target is the unaltered query.
A small discriminative head then reads the concatenated query and reconstruction and outputs a defect map. This head is trained with two kinds of examples. Negatives are real nuisance-shifted normal pairs, which teach it that residual from pose or lighting is not a defect. Positives are the synthetic defects.
The identity shortcut is the main risk: the generator could simply copy the query and ignore the references. Synthetic defects push against copying. We also measure the shortcut directly by corrupting the references at test time, as described in the decisive checks below.
The analysis goes beyond headline scores. We will measure how much residual each nuisance type induces on normal-only pairs. The comparison is across three conditions: no reference, a reference from a matched condition, and a reference from a mismatched condition. We will also stratify results by RobustAD's shift categories (position, rotation, scale, lighting, background) to see which shifts the conditioning actually cancels.
Baselines
The naive reference point is DDAD trained on the same normal images without references, run through the authors' pipeline with its 10-step setting. It shows what a strong current reconstructor does under physical shift. The file gives its tunnel result, 71.89% pixel AUROC, which we will reproduce. We also add nuisance augmentation to its training at the same budget, so that any gain we show cannot be attributed to extra data diversity.
The strongest existing competitor on the physical-shift question is PatchCore. On RAD it reports 0.833 average image AUROC, ahead of every 3D and vision-language method. Robust AD calls it the most robust unsupervised model. The per-model RobustAD numbers were not transcribed, so there is no reported result found on Lacuna for PatchCore's exact RobustAD AUROC or ARD, and we will rerun it. To be fair, PatchCore's memory bank will receive every normal image our method sees, including all nuisance-varied captures and the references. Beating it means higher image AUROC than 0.833 on RAD, and a lower Average Relative Drop than PatchCore on RobustAD. The Average Relative Drop is RobustAD's measure of how much performance falls from source to target conditions. PatchCore offers no healed image, so we also compare pixel AUROC.
For the unaligned setting, ReinADNet is the reference-based competitor. ReinAD reports 64.5% one-shot image AP for it, against a best baseline of 59.5% AP and a best baseline AUROC of 69.0%. We run our model in the same one-shot protocol, with the same references per query. Beating it means exceeding 64.5% AP at lower compute, which we report as FPS on identical hardware.
We also include FiCo as the style-decoupling baseline. It reports 97.6% AUROC on corrupted MVTec AD in its paper. On the tunnel data, MSFlow at 92.22% pixel AUROC is the bar, per Marc et al..
The key ablation isolates reference conditioning. We keep the architecture, synthetic defects, and discriminative head fixed, and replace the references in three ways: with blank inputs, with references of the wrong category, and with references from the same condition as the query. A second ablation removes the real nuisance-pair negatives from the head's training, which separates the contribution of the generator from that of the head.
Why not just…?
Why not just use PatchCore, which already wins on RAD and RobustAD? This is the most serious threat. It is possible that feature matching with a sufficiently diverse memory bank absorbs most physical nuisance. The file offers some evidence against it. On ReinAD's unaligned parts, retrieval-style baselines sit at 69.0% AUROC, and FiCo outperforms the feature method PNI by 19.4% under blur and noise. Still, neither result is a direct test on pose-varied RAD. The head-to-head comparison with a matched memory bank described above settles it. If PatchCore matches us on image AUROC, our remaining claim shrinks to localization quality and interpretable healed images, and we will say so.
Why not just register the query to a reference and run an ordinary reconstructor? Registration followed by subtraction is exactly what the 3D Gaussian Splatting pipelines on RAD do, and their image AUROC of 0.524 to 0.634 shows that small registration and rendering errors dominate the score. We will still run a 2D version, homography alignment followed by DDAD, on RobustAD's rotation, position, and scale subsets. That tells us whether learned conditioning beats explicit alignment where alignment is easiest.
Why not just train DDAD or DRÆM with heavy nuisance augmentation? Robust AD reports that synthetic corruptions are poor proxies for physical shift. Even trained on real varied normals, an unconditioned model must pick one "normal" per input with no knowledge of the current condition. The augmented-DDAD baseline, trained on the same real captures as our method, tests this directly. If it closes the gap, reference conditioning is not the active ingredient, and the ablation will show it.
Proposed Experiments
The primary benchmarks are RAD, RobustAD, and ReinAD. We report image AUROC, pixel AUROC and PRO where masks exist, the Average Relative Drop on RobustAD, and FPS measured on the same GPU as DiffusionAD's reported 23.5 FPS. The tunnel wall dataset is a stress test for stochastic textures, using neighbouring sane crops as references. Clean MVTec AD and VisA are sanity checks that conditioning does not hurt aligned performance, measured against DDAD's reported 99.8% and 98.9%.
Three decisive checks determine whether the idea holds. First, on RAD the gap between pixel and image AUROC must shrink relative to PIAD, with image AUROC above PatchCore's 0.833. Second, shuffling or blanking the references must measurably degrade results. Without that drop, the model is not using the references, and the claim fails. Third, on RobustAD's pose subsets, the residual measured on normal-only pairs must fall substantially compared with augmented DDAD.
A negative result on the tunnel data would be informative rather than fatal. It would indicate that texture stochasticity is a different failure mode from frame mismatch, and that it needs a different fix.
References
- ReinAD: Towards Real-world Industrial Anomaly Detection with a Comprehensive Contrastive Dataset
- Robust AD: A Real World Benchmark Dataset For Robustness in Industrial Anomaly Detection
- Evaluation of Unsupervised Anomaly Detection Methods on Engineering Structures
- RAD: A Dataset and Benchmark for Real-Life Anomaly Detection with Robotic Observations
- Implicit 3D Orientation Learning for 6D Object Detection from RGB Images
- Filter or Compensate: Towards Invariant Representation from Distribution Shift for Anomaly Detection
- DiffusionAD: Denoising Diffusion for Anomaly Detection
- DRÆM - A discriminatively trained reconstruction embedding for surface anomaly detection
- Anomaly Detection with Conditioned Denoising Diffusion Models