Looking Closer Where Coarse Features Cannot Explain Fine Ones

High-resolution anomaly detectors face a trade-off. Scoring every pixel at full resolution is expensive. Downsampling first is cheap, but it erases the hairline cracks and tiny foreign objects that inspection exists to catch. The usual fix, borrowed from small-object detection, is a cheap coarse pass that decides where to spend expensive high-resolution compute. The weakness of that fix is simple: the coarse pass can only point at things it can already see. We propose a different trigger, learned only from normal data. A small network is trained to predict an image's fine-scale features from its coarse-scale features. Wherever that prediction fails, the image contains detail its coarse summary cannot account for, so that region is sent for full-resolution anomaly scoring at the scale where the prediction broke.

The same signal also serves as an explanation. For each flagged region, the system can report where it looked closely, at which scale the fine structure stopped being predictable, and which parts of the image it never examined at full resolution. That last item matters because the gate itself is otherwise a black box. When sparse detectors and anomaly-focused samplers miss something, they miss it silently. We want a detector whose blind spots are measurable, and whose compute saving comes with a reported miss rate as a function of defect size.

Core Claim

The claim is that a gate trained on normal data to predict fine features from coarse ones will send small anomalies to high resolution much more reliably than a gate built on the coarse anomaly score or on objectness. The advantage should be largest for defects smaller than the coarse feature stride. As a result, sparse high-resolution scoring should match dense full-resolution scoring on localization while spending a fraction of the compute.

Write $f_1$ for the early-stage (high-resolution) backbone features, $f_3$ for the stage-3 (coarse) features, $\uparrow$ for bilinear upsampling, and $g$ for a lightweight decoder trained only on normal images. The escalation signal at location $x$ is

$$r(x) = \lVert f_1(x) - g(\uparrow f_3)(x) \rVert^2 .$$

Early-stage features at full resolution are cheap, since they are a thin, shallow computation. The expensive part is dense density estimation and deep processing at that resolution, and only the tiles with high $r$ receive it.

This is not "attach QueryDet to an anomaly detector." QueryDet predicts where small objects are from coarse pyramid levels and restricts high-resolution computation to those locations. ESOD predicts class-agnostic objectness right after the backbone's first layers and slices feature maps to keep only target-dense patches. Both gates are supervised on known object categories, and both inherit the failure the file names explicitly: a target that leaves no trace on the coarse map is never examined. Our gate needs no anomaly labels. It measures how much information was lost between scales, not whether something looks like a known object.

Where This Sits in Prior Work

The motivation comes from three findings that the resolution loss is real and structural. SSD notes that after 32× downsampling, a 10-pixel object becomes a sub-pixel activation. Finding Tiny Faces shows that tiny targets must be recognized mostly from their surroundings, because they carry almost no internal texture. ContextMix reports that small industrial defects are blurred or eliminated by the time features reach deep detection heads.

The sharpest constraint comes from MSFlow. MSFlow runs parallel normalizing flows (density models that map features to a simple distribution) over ResNet stages 1 to 3 and fuses them across scales. Its authors report that on minute defects such as MVTec AD screws, the fused receptive field overwhelms the local signal. This rules out one design: dense cross-scale fusion that lets coarse context vote on every pixel. We instead score each escalated tile at the scale where it was escalated.

LIP shows that a learned importance map during pooling preserves small-object detail, raising Faster R-CNN small-object AP on COCO from 21.9% to 24.0%. We borrow its lesson that the decision about what survives downsampling should be learned. We differ in two ways. Our map is learned from normal data only. It also decides where to compute, rather than how to pool.

Holmes-VAU is the anomaly-side analogue of sparse compute. It uses a lightweight scorer to pick 8 to 16 keyframes for a multimodal language model. Its authors, and the file, note that faint anomalies scored low by the cheap pass are filtered out before the expensive model ever sees them. That is our failure mode on the time axis, and here we address it on the spatial axis.

CAMERAS makes high-resolution explanations by fusing saliency maps from seven input scales. That costs seven forward and backward passes. Our scale-resolved explanation is a by-product of the gate and needs no extra passes.

Research Program

Build the gate. The system has four pieces. The first is a frozen ImageNet ResNet backbone, as in MSFlow. The second is the decoder $g$, trained on normal training images to predict stage-1 features from upsampled stage-3 features. We will also train a stage-2 variant, so that the scale at which prediction first fails can be reported. The third is a tile-level escalation rule that keeps the top fraction $\rho$ of tiles ranked by pooled $r$. The fourth is a full-resolution scorer that runs only on kept tiles. That scorer is MSFlow's stage-1 flow applied to native-resolution tiles, executed sparsely in the style of ESOD's retained-location computation. Non-escalated tiles keep their coarse-scale MSFlow score.

Calibrate the miss rate. The escalation budget $\rho$ is chosen on a held-out split containing synthetic anomalies pasted at controlled sizes. For each $\rho$, we measure escalation recall: the fraction of anomalous pixels that land in escalated tiles. The explanation report for each test image then includes three things. It gives the anomaly map. It gives the scale at which each flagged region became unpredictable. It gives the uncovered area together with the calibrated recall for defects of each size band.

One caveat is known in advance. Synthetic anomalies are not real ones, so the calibration is checked against real test masks rather than trusted blindly.

Baselines

Naive reference. The do-nothing point is MSFlow run as published, at its standard downsampled input resolution, with dense fusion. The file reports 99.7% detection AUROC, 98.8% localization AUROC, and 97.1% PRO on MVTec AD for this model (MSFlow). No per-class screw number was found on Lacuna, so we will rerun it to obtain per-class and per-defect-size results. On VisA, no reported result was found on Lacuna for MSFlow.

Accuracy ceiling. The strongest competitor is dense MSFlow re-run at native image resolution. It represents the accuracy that paying full compute buys, and our method has to approach it at lower cost.

Coarse-score gate. This is the most important alternative, and it isolates our key ingredient. It uses the same sparse machinery and the same budget $\rho$, but ranks tiles by MSFlow's coarse anomaly score. This is the spatial analogue of Holmes-VAU's sampler.

Objectness-style gate. A class-agnostic objectness head in ESOD's style is trained on normal images to fire on structured foreground.

Cheap high-frequency gate. Tiles are ranked by raw-pixel Laplacian energy.

Fair comparison. All gates share the backbone, the high-resolution scorer, the tile size, and the same set of $\rho$ values. All receive equal hyperparameter search on the synthetic-anomaly split. Compute is measured as GFLOPs and wall-clock time on the same GPU.

What counts as winning. Beating the field means two things. At equal $\rho$, our gate must achieve higher escalation recall and higher PRO than every other gate on defects smaller than the stage-3 stride. At some $\rho$ well below 1, our PRO must fall within noise of dense native-resolution MSFlow.

Why not just…?

Why not just run MSFlow densely at full resolution? If compute is free, this is the accuracy ceiling and we expect it to win or tie. Two things keep the question open. Dense native-resolution flows multiply cost roughly with image area; QueryDet reports that adding a dense high-resolution level alone cuts throughput from 13.6 to 4.85 FPS. MSFlow's own screw failure also suggests that dense fusion can hurt tiny defects rather than help them. The experiment measures both effects directly. If dense native resolution also fixes the small-defect deficit, our contribution reduces to compute savings and must be judged on those alone.

Why not just gate on the coarse anomaly score? This is the real threat. If coarse MSFlow scores already rank the right tiles, the cross-scale predictor adds nothing. Our prediction is specific: the two gates tie on large defects, but the coarse-score gate's escalation recall collapses once defects are smaller than the coarse stride, as the SSD sub-pixel argument implies. If that collapse does not appear in the recall-versus-size curves, we drop the claim.

Why not just distill multi-scale testing into one scale, as MSCD does? Multi-Scale Cross Distillation recovers multi-scale accuracy at single-scale speed for two-stage box detectors on DOTA, improving its baseline by 6.50 mAP. However, the file records that such distillation struggles to keep fine boundaries for extreme small targets. It also offers no per-image account of what was missed. We include a distilled single-scale flow as an extra comparison on the smallest defect band.

Why not just use Laplacian or edge energy? It costs almost nothing. The problem is that textured normal surfaces are full of high-frequency detail, so the budget would be spent on normal texture. The high-frequency gate baseline tests this directly on textured categories.

Proposed Experiments

Main benchmarks. We use MVTec AD and VisA at native resolution. Results are reported per category and in defect-area bands computed from the ground-truth masks. Metrics are image AUROC, pixel AUROC, PRO, escalation recall, GFLOPs, and latency, each plotted against $\rho$.

Decisive check. The key figure plots escalation recall against defect area for all gates at a fixed $\rho$ (for example, 0.2). Our method succeeds only if its curve stays flat into the band below the coarse stride while the coarse-score gate's curve drops.

Explanation checks. First, we test whether the reported failure scale correlates with true defect size. Second, we test whether the calibrated miss rates from synthetic anomalies hold on real masks.

Transfer probe. We apply the same gate, trained on normal frames, to UCFCrime2Local. The comparison point is Anomaly Locality in Video Surveillance, where oracle tubes reached 74.73% AUC against 56.12% for full frames. The question is whether our gate finds anomalous locations without precomputed boxes.