Many sensors do not record an image. A single-pixel camera, an accelerated MRI scanner, or an in-sensor compressing chip records a short vector of linear projections instead, often ten to a hundred times shorter than the image it encodes. When the goal is only to decide something, such as which class, defective or not, or tumor or not, a mature line of theory shows that the image is unnecessary. A decision needs far fewer measurements than a faithful picture. That theory, however, covers only the closed-world case, where every class or target signature is known in advance. Real inspection and screening problems are open-world. The defect or lesion has never been seen, and only normal examples exist for training. The standard tool for that setting is reconstruction-based anomaly detection: train a model only on normal data, let it "heal" an input back to normal, and flag whatever it had to change. This tool assumes a full image.
We propose to join the two. We train a fast network that operates entirely on the measurement vector and maps any measurement to the nearest measurement a normal object could have produced. We then score the anomaly as the whitened leftover, which is the part of the measurement a normal-only model cannot explain. The central claim is concrete and falsifiable: the number of measurements needed for reliable detection is set by the intrinsic dimension of the normal data, not by image size or defect size. That number sits well below the budget needed to reconstruct an image good enough for a downstream detector. A second claim is that the obvious alternative, reconstructing with a normal-trained network and then detecting, fails for a specific reason. The healing erases the very anomaly the downstream detector needs. The measurement residual retains that anomaly.
Core Claim
Let the sensor produce $y = \Phi x + w$, where $x \in \mathbb{R}^N$ is the unseen image, $\Phi \in \mathbb{R}^{M \times N}$ is the known sensing matrix with $M \ll N$, and $w$ is noise. We learn a projector $G_\Phi: \mathbb{R}^M \to \mathbb{R}^M$ from normal data only, and score an input by
$$s(y) = \| W\,(y - G_\Phi(y)) \|_2^2,$$
where $W$ is a whitening matrix estimated from residuals on held-out normal measurements. The linear case shows why this should work and where it breaks. Suppose normal images lie in an $r$-dimensional subspace with basis $U$, and an anomaly adds a component $a$. The residual after projecting onto the span of $\Phi U$ is then $P^\perp \Phi a$. For a generic random $\Phi$, this residual is nonzero once $M \geq r+1$, whatever $N$ is and wherever the defect sits. This extends the $M = r_\Sigma + 1$ phase transition that Bounds on the Number of Measurements for Reliable Compressive Classification proved for closed-world subspace classification to the one-class, unknown-anomaly case. We do not expect a clean closed form for curved image manifolds. We do expect, and will test, that the empirical detection threshold $M^\ast$ tracks an estimate of the local intrinsic dimension of normal data, and that $M^\ast$ falls below the measurement rate at which reconstruct-then-detect becomes competitive.
The prior work plays three different roles here. Target detection performance bounds in compressive imaging motivates the idea. It detects anomalous hyperspectral pixels directly from compressed data with whitened nearest-neighbor rules against a linear library of background spectra, and it shows error decaying exponentially in measurement count. We replace the linear library with a learned nonlinear normal model over spatial patterns. Compressive Sampling for Signal Detection constrains the design. It shows that blind energy detection on compressed data is badly suboptimal, and that its alternative of template matching needs a known target. A usable score therefore needs a structured prior on the normal class, since no prior on the anomaly is available. The MRI Scanner as a Diagnostic: Image-less Active Sampling shows that image-less diagnosis from undersampled k-space (the raw Fourier-domain data an MRI scanner records) is clinically viable. Its method is supervised on a single labeled pathology and returns one label per slice, whereas ours is unsupervised and returns a coarse location. InvAD: Inversion-based Reconstruction-Free Anomaly Detection with Diffusion Models is the closest in spirit. It avoids image-space denoising by inverting a diffusion model in the feature space of a pretrained EfficientNet-B4. In that work, "reconstruction-free" still means a fully sampled RGB image passes through an ImageNet backbone. Our input never becomes an image.
Research Program
The training recipe comes from DRÆM - A discriminatively trained reconstruction embedding for surface anomaly detection. DRÆM pastes synthetic anomalies, built from Perlin-noise masks filled with texture patches, onto normal images, and trains a network to remove them. This keeps the network from learning to copy its input, a failure known as the identity shortcut. Linearity of the sensor makes the recipe cheap in our setting, because a synthetic anomaly $a$ enters the measurement as the additive term $\Phi a$. We train $G_\Phi$ to satisfy $G_\Phi(\Phi(x + a) + w) \approx \Phi x$ on normal $x$ and to leave clean normal measurements unchanged. A copying solution is therefore penalized directly. The same training also removes the "Goldilocks" noise-level tuning that diffusion-based detectors need, because there is no forward noising step. $G_\Phi$ is a small feedforward network, and inference is a single pass.
Localization is the second ingredient, and it comes from the sensing design rather than from a decoder. We use block-wise sensing, in which each 33×33 image block gets its own set of measurements, following the block-based simulation protocol of ReconNet: Non-Iterative Reconstruction of Images from Compressively Sensed Measurements. The network shares context across blocks, so the normal model can use global object structure. The score is computed per block, which yields a coarse anomaly map at no extra cost. A finer map within each block comes from back-projecting the whitened residual through $\Phi^\top$. We will report how much localization quality is lost at block resolution rather than hide it inside pixel-level metrics.
The third component is analysis. For each category we estimate the local intrinsic dimension of normal blocks from the singular-value decay of neighborhood patches. We then measure the detection AUROC as a function of the measurement rate $M/N$ and test whether the rate where AUROC saturates tracks that dimension. Texture categories and object categories should differ in a predictable direction.
Baselines
The naive reference point is a linear one-class model on the measurements: a per-block Gaussian fit to normal $y$, scored by the Mahalanobis residual outside its top principal components. This is the direct descendant of the whitened dictionary rules in Target detection performance bounds in compressive imaging, and it is exactly the linear case of our claim. It uses the same measurements and the same normal training set, and its only tuned quantity, the number of components, is chosen on held-out normal data. We beat it only if the nonlinear projector gives a clear image-level AUROC gain at the low measurement rates, where curvature of the normal manifold should matter most. Plain energy detection on $\|y\|$ is included as a floor.
The strongest real competitor is reconstruct-then-detect. We train a ReconNet-style feedforward reconstructor at each measurement rate and feed its output to the strongest ambient detectors in the file. These are FastFlow, which Benchmarking Unsupervised Anomaly Detection and Localization reports at 99.4% image-level AUROC and 98.6% pixel-level AUROC at 21.8 FPS on full MVTec AD images; InvAD, which reports 99.0% AU-ROC at 88.1 FPS on MVTec AD and 96.9% on VisA; and DRÆM, which reports 98.0 image AUROC and 68.4 pixel AP on MVTec AD. The reconstructor is trained in two variants: on generic natural images, and on the normal training images of each category. The downstream detectors are retrained on reconstructions from normal inputs, so they see reconstruction artifacts at training time. Our method and this pipeline get the same GPU and a matched hyperparameter search budget, and every reported speed includes the reconstruction step. The full-image numbers above serve as an oracle ceiling, not a target. Beating this baseline means higher image-level AUROC at equal measurement rate for rates at or below 10%, or equal AUROC at higher throughput. If reconstruct-then-detect wins at every rate, the claim is wrong.
For MRI, no reported result was found on Lacuna for unsupervised anomaly detection from undersampled fastMRI k-space. The closest number is the supervised meniscus-tear AUC of 0.780 at 7% of k-space from The MRI Scanner as a Diagnostic: Image-less Active Sampling. We will report our unsupervised score on that same task at 7% and 25% as a reference point. We do not expect to match a supervised classifier. The question is how much of that gap label-free training closes.
Three ablations isolate the key ingredients. The first removes synthetic anomalies and trains $G_\Phi$ as a plain autoencoder on normal measurements, to show the identity shortcut. The second drops whitening, to show that raw residuals are dominated by normal variation in high-variance measurements. The third replaces the learned projector with measurement-consistent projection through an ambient generator trained on normal images, which is slower but more principled. This separates "learned in measurement space" from "uses a normal prior at all."
Why not just…?
Why not just reconstruct the image and run PatchCore or FastFlow? This is the real threat, and at high measurement rates it will probably win, because a good reconstruction plus a strong pretrained detector is hard to beat. At low rates it faces a dilemma. A reconstructor trained on generic images smooths fine structure, and fastMRI+: Clinical Pathology Annotations for Knee and Brain Fully Sampled Multi-Coil MRI Data documents that fidelity-optimized MRI reconstructions blur or hallucinate small anatomy. A reconstructor trained on normal images does worse in a specific way: it heals the defect, and the detector then sees a normal image. Our score uses that healing instead of fighting it. The healed estimate disagrees with $y$, and that disagreement is the signal. The two-variant reconstruct-then-detect baseline tests this directly. We predict the normal-trained variant will be worst at low rates, and if it is not, the argument fails.
Why not just fit a Gaussian or a density model to the measurements? For global anomalies this may be enough, and our linear baseline is exactly this model. A small local defect spreads its energy thinly across a block's measurements, while normal pose and texture variation is large and nonlinear. A single Gaussian therefore places the defect inside its high-variance directions. Pure likelihood models add the known problem that simple, low-entropy inputs receive high probability, as discussed in Anomaly Detection via Generative Manifold Projection. If the Mahalanobis baseline matches the learned projector on MVTec objects, we will conclude that the nonlinear model is unnecessary and report that.
Why not just capture a low-resolution image? Target detection performance bounds in compressive imaging found that random projections beat spatial downsampling for hyperspectral targets, because binning discards high-frequency detail. That result concerns spectra, not spatial defects. For large defects on smooth objects, binning plus an off-the-shelf detector could be competitive, and it has the advantage of working with pretrained backbones. We will run binning at equal $M$ with FastFlow on the binned image, broken down by defect size. We expect random sensing to win on thin scratches and small spots and to lose little or nothing on large defects.
Proposed Experiments
The main benchmark is MVTec AD, with 15 categories and pixel-annotated defects (MVTec AD - A Comprehensive Real-World Dataset for Unsupervised Anomaly Detection). We simulate block compressive sensing with Gaussian and very sparse random matrices, the latter following Compressive detection with sparse random projections, at measurement rates of 1%, 4%, 10%, 25%, and 50%. VisA serves as a second industrial test. For a physically native compressed domain, we use fastMRI single-coil knee k-space (fastMRI: An Open Dataset and Benchmarks for Accelerated MRI) with retrospective Cartesian undersampling. We train on slices carrying no fastMRI+ pathology label and test on labeled slices, using fastMRI+ bounding boxes for localization. Here the "blocks" become image regions reached through the adjoint of the undersampled Fourier operator. We treat unlabeled slices as normal only provisionally, because missing annotations are likely. We will measure the effect by holding out slices that radiologists labeled as normal. We report image-level AUROC, block-level AUROC, pixel AUROC and AP after upsampling the block maps, and throughput including all preprocessing.
Three checks decide the proposal. The first is a crossover plot of AUROC against measurement rate for our method and for reconstruct-then-detect. The claim survives only if a range of low rates exists where the measurement residual wins. The second is the dimension test. Across the 15 MVTec categories, the rate at which our AUROC reaches 95% of its 50%-rate value should correlate with the estimated intrinsic dimension of normal blocks. It should not correlate with defect area, which the linear analysis says should not matter. The third is the healing test. On anomalous test images, we measure how much of the true defect survives in the normal-trained reconstruction, and whether that erased fraction predicts where the reconstruct-then-detect baseline loses to the measurement residual. If all three hold, the result is a detector that never forms an image, a principled rule for how many measurements it needs, and an explanation of why the obvious pipeline fails. If the crossover never appears, the finding is also useful: for open-world detection, fast reconstruction is enough, and compressed-domain inference should stay closed-world.
References
- Bounds on the Number of Measurements for Reliable Compressive Classification
- Target detection performance bounds in compressive imaging
- Compressive Sampling for Signal Detection
- The MRI Scanner as a Diagnostic: Image-Less Active Sampling
- InvAD: Inversion-based Reconstruction-Free Anomaly Detection with Diffusion Models
- DRÆM - A discriminatively trained reconstruction embedding for surface anomaly detection
- ReconNet: Non-Iterative Reconstruction of Images from Compressively Sensed Measurements
- Benchmarking Unsupervised Anomaly Detection and Localization
- fastMRI+: Clinical Pathology Annotations for Knee and Brain Fully Sampled Multi-Coil MRI Data
- Unsupervised Out-of-Distribution Detection with Diffusion Inpainting
- MVTec AD - A Comprehensive Real-World Dataset for Unsupervised Anomaly Detection
- Compressive detection with sparse random projections
- fastMRI: An Open Dataset and Benchmarks for Accelerated MRI.