Visibility-Weighted Box Regression for Normal-Data-Only Defect Detection

Factory inspection systems usually report defects as boxes, but the anomaly detectors that can catch defects never seen before work differently. These are reconstruction-based detectors: models trained only on defect-free images that "repair" a test image into its nearest normal version and flag the places where input and repair differ. They output pixel heat maps, not boxes. Today, turning those heat maps into boxes means picking a threshold and drawing rectangles around connected blobs. That is the step where these methods are weakest. With no labelled defects available for tuning, the threshold cannot be set reliably, and classical generative detectors can score 0.86 image-level AUROC on cables while reaching only 0.05 per-region overlap (MVTec AD). This note proposes training a box-regression head directly on the input–reconstruction pair, supervised only by synthetic defects pasted onto normal images. The proposal centres on one design choice that existing box losses get wrong in this setting.

The design choice concerns which boxes to trust. Losses that reweight boxes, such as Focal-EIoU, decide how much each training box counts from how well the prediction currently overlaps its target. When the target comes from a synthetic defect, much of the regression error comes from the target, not from the model. Pasted textures blend into the background at their fringes, and the edges of a mask-derived box often lie on pixels that no detector could see. The synthesis procedure knows exactly how visible each edge is. We propose to weight, and optionally pull inward, each box edge by its visibility at generation time. We will test whether this beats overlap-based weighting, learned-uncertainty losses, and threshold-plus-blobs on box-level localisation of real defects.

Core Claim

In self-supervised anomaly box regression, the dominant source of regression noise is ambiguity in the synthetic targets. That ambiguity is known exactly when the anomaly is generated. A loss weighted by per-edge visibility should therefore localise real defects better than a loss weighted by prediction quality. The predicted advantage should be largest on small and low-contrast defects. There, overlap-based weighting is known to misfire, because small pixel shifts drop the overlap sharply and push small targets into the "low quality" bin (Focal and efficient IOU loss). Visibility does not depend on object size, so the weighting should not penalise small defects in the same way.

Concretely, synthesis produces a mask $m$ and a per-pixel blend opacity $\alpha$. This follows DRÆM's Perlin-mask texture pasting (DRÆM) or TransFusion's transparency blending (TransFusion). For each of the four box edges $k$, we compute a visibility score $v_k \in [0,1]$. It is the mean, over a thin strip just inside that edge, of $\alpha$ times the local contrast between the corrupted image and the clean original. The regression loss for one box is

$$\mathcal{L} = \sum_{k=1}^{4} \frac{v_k}{\bar v}\,\ell\big(\hat t_k - t_k(v)\big) + \mathcal{L}_{\text{IoU}},$$

where $\hat t_k$ is the predicted edge offset and $\ell$ is a smooth-$L_1$ penalty. The target edge $t_k(v)$ is the mask edge, optionally moved inward to the outermost strip whose visibility exceeds a floor. The normaliser $\bar v$ is the batch-mean visibility, which keeps the overall gradient scale fixed. $\mathcal{L}_{\text{IoU}}$ is an unweighted overlap term that keeps whole-box geometry coupled.

Relation to Prior Work

Two works sit closest to this proposal, and both motivate it. WeakREST (Efficient Anomaly Detection with Budget Annotation) combines patch residuals with boxes, but only as human-drawn masks that mark outside blocks as normal and inside blocks as unknown. It never regresses box coordinates, and it needs human annotation. The improved YOLOv8 for photovoltaic cells (Defect Detection of Photovoltaic Cells) regresses boxes with Focal-EIoU and reaches 86.6% [email protected] on PVEL-AD-2021. It is fully supervised and closed-set, so a defect type absent from its 12 training classes is simply missed. We keep WeakREST's goal of cheap supervision but remove the human boxes entirely. We keep the photovoltaic detector's regression head but feed it residual evidence that does not depend on defect class.

The reconstruction side supplies the method. DRÆM shows that a discriminative network reading the concatenated input and reconstruction learns a better difference measure than subtraction does. Our box head attaches to that same concatenation. TransFusion supplies a stronger but slower reconstructor: 0.34 s per sample versus DRÆM's 0.05 s. It also supplies an explicit transparency factor that maps directly onto our $\alpha$.

The weighted-regression literature both supplies and constrains the design. Three papers disagree about which boxes should drive regression. Focal-EIoU up-weights high-overlap boxes. HAMBox (HAMBox) finds that over 80% of correct face detections come from anchors that started below 0.5 overlap. Focaler-IoU (Pupil Center Localization with Improved YOLOv8) focuses on moderate overlap. All three weight by prediction quality, which mixes up "hard but learnable" with "unlearnable because the target is wrong." Our weighting uses a different signal that only exists for synthetic data.

KL loss with variance voting (Bounding Box Regression With Uncertainty) is the strongest conceptual competitor. It lets the network learn per-edge variance, so it down-weights ambiguous edges without being told which ones they are, and it improved strict $AP^{90}$ by 6.2% on COCO. Our hypothesis is that learned variance on synthetic data mostly absorbs visibility, and that supplying visibility directly is cheaper and more robust. If this hypothesis is wrong, it falsifies the proposal, and we test it head on. Automated Focal Loss (Automated Focal Loss) is relevant as a tuning-free alternative to fixed focusing exponents. We include it so that our method is not credited simply for avoiding hyperparameter search.

Research Program

We will build a single-pass system with three parts. First, a reconstructive network in the DRÆM style, trained on normal images corrupted with synthetic anomalies. Second, a discriminative trunk on the input–reconstruction concatenation. Third, an anchor-free detection head on that trunk that predicts an objectness score and four edge offsets per location.

The synthesis pipeline will be instrumented so that every pasted anomaly records its mask, opacity field, and per-edge visibility. Fragmented synthetic anomalies, such as several Perlin islands, will be assigned boxes through the same merging rule we later use to turn real ground-truth masks into evaluation boxes. This keeps the training and evaluation definitions of "one defect" consistent.

We will train four variants on identical synthetic streams. They differ only in the regression loss: unweighted smooth-$L_1$ plus IoU, Focal-EIoU, KL loss, and visibility weighting. Visibility weighting is run both with and without target shrinking. A fifth variant swaps the DRÆM reconstructor for TransFusion, to see whether better reconstruction narrows or widens the gap between losses.

The analysis then breaks localisation error down by real-defect size and contrast on MVTec AD and VisA. It also measures how strongly each variant's learned variance, if it has one, correlates with the visibility we compute.

Baselines

The naive reference is threshold-plus-blobs. We take DRÆM's anomaly map and TransFusion's mask, binarise each, take connected components, and draw their bounding rectangles. The threshold is chosen in two ways. One is an honest choice from synthetic anomalies on held-out normal images. The other is an oracle choice using real anomalies, which serves as an upper bound. This baseline is the strongest competitor because it uses exactly the same reconstructors. Any gain from our head must come from learning to box, not from better maps.

The strongest reported numbers in the file are pixel-level, not box-level. TransFusion reports 99.2% image AUROC and 91.6% AUPRO on MVTec AD, and DRÆM reports 68.4 pixel AP. We found no reported box-level AP result on Lacuna for any method on MVTec AD or VisA, so we will compute box AP for all baselines ourselves under one protocol.

The second baseline is an off-the-shelf detector, YOLOv8s with Focal-EIoU as in the photovoltaic paper, trained on the same synthetic anomalies but seeing only the raw image with no reconstruction. This baseline isolates the value of the reconstruction input.

The third baseline is supervised detection on PVEL-AD-2021 with the paper's 86.6% [email protected] configuration. We will rerun it with defect classes held out, so that it competes on unseen defects.

The loss ablations are unweighted, Focal-EIoU, KL loss, and Automated Focal Loss. They share architecture, synthetic data, schedule, and a matched hyperparameter search of equal trial count. They isolate the visibility weighting as the key ingredient.

Beating the baselines means two things. Visibility weighting must beat the best competing loss on box AP at IoU 0.5 averaged over MVTec AD and VisA, beyond seed variance across at least three seeds. It must also beat honestly thresholded blobs from the same reconstructor.

Why not just…?

Why not just threshold the residual map and draw boxes around connected components? Bergmann et al. identify threshold selection without anomalous validation data as an unsolved problem. Blobs also fragment thin defects into many boxes or merge them with background artefacts. Still, a good reconstructor plus an oracle threshold could come close. That is why we report the oracle variant as well. If our head only matches the oracle threshold, the contribution is threshold-freeness rather than better localisation, and we will say so.

Why not just train a standard detector on synthetic anomalies, without reconstruction? This is the most serious threat, because a strong detector may learn "texture that does not belong here" on its own. The raw-image YOLOv8s baseline tests this directly. Supporting evidence for reconstruction comes from DRÆM, whose discriminative sub-network gains from seeing the reconstruction, and from DeScarGAN (DeScarGAN), which shows that structural deformations are visible mainly against a normal counterpart. If the raw detector matches ours on MVTec objects, the reconstruction component should be dropped.

Why not just use KL loss and let the network learn which edges are ambiguous? It might work equally well. If the learned variances correlate strongly with our visibility and the box AP is matched, the explicit signal is redundant. Our bet is that learned variance confounds visibility with model error early in training, and that it fails on small defects where variance estimates are noisy. The size-stratified comparison settles this question.

Why not just train a supervised detector on real defect boxes? On known classes it will probably win. The photovoltaic detector's 86.6% [email protected] is the reference. Our claim covers only unseen defect types, and the held-out-class protocol below is the test of that claim.

Proposed Experiments

On MVTec AD and VisA, ground-truth boxes are derived from pixel masks by connected components with a fixed dilation-merge rule. We will report results under two dilation radii so the conclusion does not depend on the box definition. Metrics are box AP at IoU 0.3 and 0.5, region recall at one false positive per image, and image AUROC from the maximum box score as a sanity check. Each metric is stratified by defect area tercile and contrast tercile. We also report latency, to keep the single-pass DRÆM-style variant comparable with the 20-step TransFusion variant.

On PVEL-AD-2021, we train the reconstruction system only on normal cells, assuming the dataset provides enough defect-free images; this needs checking. We hold out a subset of defect classes and compare against the supervised YOLOv8s Focal-EIoU detector trained on the remaining classes, scoring [email protected] on the held-out classes only.

The first decisive check is the small-defect tercile. Visibility weighting should beat Focal-EIoU by more there than on large defects. If the gap is flat across sizes, the mechanism claimed here is not what drives any gain.

The second decisive check is a controlled corruption test. We deliberately inflate synthetic boxes with invisible fringes. Prediction-quality losses should degrade on real-defect AP while visibility weighting stays stable. If it does not stay stable, the core claim is wrong.