A factory inspection system is rarely taught one product and left alone. New products arrive over time, old ones stay on the line, and a camera image does not come labeled with which product it shows. The system therefore has two jobs. It has to decide which learned model of "normal" applies to the image; this is the routing problem from continual learning, where the model must keep earlier skills while learning new ones. It also has to decide whether the image is defective, which is the anomaly detection problem. Current systems solve the two jobs with separate machinery. Our proposal is to use a single set of small normalizing flows for both. A normalizing flow is an invertible network that gives an exact probability for each image patch. Each product gets its own flow. A test image is routed to the flow under which most of its patches look most probable, and that same flow then flags the few patches it finds improbable as defects.
The point that makes this more than a merger of two pipelines is that routing and detection pull against each other. A defective image is, by definition, partly unlike its own product's normal data. A router that averages over the whole image can therefore send a defective part to the wrong product's model and hide the defect. We propose to route on a trimmed patch-likelihood statistic that ignores the least typical patches. These are exactly the patches the detector needs to flag. The same statistic computed on a small random subset of patches also cuts the cost of evaluating every product's flow, which otherwise grows linearly with the number of products. The test is simple: under naive routing, routing errors should concentrate on defective test images, and under trimmed routing they should disappear.
Core Claim
Assume one compact conditional flow per task, of the kind used in CFLOW-AD. That method maps patch features from a frozen pretrained backbone, conditioned on spatial position, to a Gaussian, and scores anomalies by low patch likelihood. Under that assumption, the right routing statistic for task-agnostic continual inspection is
$$ s_k(x) = \operatorname{TrimMean}_{\alpha}\big\{ \log p_k(h_p(x) \mid \mathrm{pos}_p) : p \in S \big\}, \qquad k^\star = \arg\max_k s_k(x). $$
The symbols are defined as follows. $x$ is a test image. $h_p(x)$ is the backbone feature at patch $p$. $p_k$ is the density of task $k$'s flow. $S$ is a random subset of patch locations. $\operatorname{TrimMean}_{\alpha}$ averages after discarding the fraction $\alpha$ of patches with the lowest log-likelihood. The anomaly map is then $-\log p_{k^\star}$ evaluated on all patches.
We make three claims, each separately testable. First, exact log-densities from different flows over the same feature space can be compared directly. Reconstruction errors from different autoencoders cannot, because their scale depends on each autoencoder's capacity and on the complexity of the data it was trained on. Second, trimming makes routing robust to the localized defects that the detector is supposed to find. Third, routing on $|S| \ll P$ patches, where $P$ is the total number of patches, reduces the routing cost from roughly $N \cdot P$ to $N \cdot |S|$ flow evaluations for $N$ tasks, while preserving routing accuracy. Only the selected flow is then run on all $P$ patches.
Positioning Against Prior Work
The routing half builds on the bank-of-gatekeepers line. Expert Gate routes each input to the expert whose autoencoder reconstructs it best and reports 99.2% gating accuracy on a six-task image sequence. SurpriseNet explicitly casts task inference as anomaly detection. Its pixel autoencoders collapse to 16.49% on Split CIFAR-100, and it recovers to 48.15% only by placing the autoencoders on frozen ResNet-18 features. That failure motivates our choice. It shows that comparing unnormalized reconstruction errors across task models breaks on complex images. It also shows that frozen features are effectively required, so we adopt them from the start.
HCL is the closest flow-based router. It fits whole-image RealNVP and Glow flows per task, routes by Bayes' rule on exact likelihood, and detects task changes from likelihood typicality. It was evaluated only on low-resolution classification benchmarks, scores whole images rather than patches, and never faces inputs that are partially anomalous by design. Our proposal differs on all three points.
Two results constrain the design. A Geometric Explanation of the Likelihood OOD Detection Paradox shows that a Fashion-MNIST flow scores MNIST with an AUROC of 0.07, because simpler data lies on lower-dimensional regions with high density. This warns against absolute likelihood thresholds. We therefore route by comparing experts on the same input, so the complexity of that input largely cancels. Where an absolute decision is needed, for deciding whether an image belongs to a new task, we use per-expert quantiles calibrated on held-out normal data. We test whether the paradox reappears in that step. The second constraint comes from Distilling Normalizing Flows and CFLOW-AD's analysis of sequential coupling layers: flows are slow because their layers must run in sequence. That is why we reduce the number of patches each expert must score rather than adding flow depth.
The detection half is set against single-task detectors. AE-FLOW places a flow inside an autoencoder bottleneck for medical images, and AST pairs a flow teacher with a feed-forward student. Both are strong on one dataset, and neither contains any mechanism for multiple tasks.
The direct competitor is UCAD. It runs task-agnostic continual anomaly detection on MVTec AD and VisA by matching each image to stored feature keys from a frozen Vision Transformer. It then scores the image against a nearest-neighbor bank of normal features, adapted by learned prompts and SAM-derived masks. Its routing statistic and its anomaly score come from different mechanisms. Ours share one density model, which is what lets us analyze and control how defects disturb routing.
Generate to Discriminate trains a separate classifier on synthetic samples to act as router. It shows that discriminative routing can beat rehearsal, but it requires a heavy generator per domain. We avoid both the extra router and the generator.
Research Program
We build the system on a frozen WideResNet-50 backbone with multi-scale patch features, following CFLOW-AD. Each new task receives its own CFLOW-style decoder, trained only on that task's normal images. After training, the decoder is frozen. This parameter isolation removes forgetting of the density models by construction, so the only way old tasks can degrade is through misrouting. That property is deliberate: it lets us attribute every accuracy loss to the router.
The analysis has three parts. The first is a misrouting audit. For each routing statistic, we measure routing accuracy separately on normal and defective test images, and relate errors to the area of the defect mask. The second is a cost study that sweeps the patch budget $|S|$ and the number of tasks $N$, measuring throughput and routing accuracy together. We also test whether evaluating only one coarse feature scale for routing is enough. The third is a task-free variant. Here a new task is spawned when even the best expert finds fewer than a calibrated fraction of patches typical. The hypothesis is that a defective image of a known product leaves most patches typical under its own expert, while an image of an unseen product leaves almost none typical. If that holds, the fraction of typical patches should separate "new product" from "known product with a defect" better than mean likelihood does.
Baselines
The naive reference point is a single CFLOW-AD decoder fine-tuned sequentially on each new category with no routing. It measures how much forgetting there is to fix. Its counterpart is the oracle: CFLOW-AD given the true task identity. This bounds what any router can achieve, and the gap between our system and the oracle is our main error measure. The relevant single-task numbers are the MVTec AD figures reported by CFLOW-AD: 98.26% image AUROC and 98.62% pixel AUROC. We will rerun the oracle ourselves under our backbone and training budget rather than rely on the reported figures.
The strongest existing method is UCAD. For the continual MVTec AD stream it reports 93.0% image AUROC, a forgetting measure of 0.010, and roughly 23 MB of memory for 15 tasks; on VisA it reports 87.4% image AUROC. We will run it with its released protocol, using the same task order, the same test splits, and the same GPU for timing. Beating it means exceeding its image AUROC on both streams with no worse forgetting, at comparable memory. We will report the total size of our per-task flow bank against UCAD's 23 MB. If our memory cost is higher, we will say so.
Three ablations isolate the key ingredient by swapping out only the routing statistic while keeping the per-task flows and the anomaly scorer fixed. The first routes by the untrimmed mean patch likelihood, an HCL-style statistic applied to patches. The second routes by minimum feature-reconstruction error from per-task autoencoders on the same frozen features. This is the SurpriseNetE and Expert Gate approach, and it tests whether exact densities actually beat reconstruction for routing. The third routes by UCAD-style nearest-key matching on global image features. Every variant uses the same budget of patches, training epochs, and hyperparameter search trials on a held-out validation split of normal images. No defect labels are used for tuning.
Why not just…?
Why not just route with global image keys, as UCAD does? This is the most serious threat. MVTec categories are visually distinct, so global keys may already route nearly perfectly, and a global descriptor is naturally insensitive to small defects. If the misrouting audit shows near-perfect key routing on both normal and defective images, the routing claim fails on MVTec AD. The contribution would then shrink to the cost and memory profile of a unified flow system plus the task-free spawn test. To find out whether a harder regime exists, we will run the audit on VisA's groups of closely related categories, where global keys are more likely to confuse tasks. We commit to reporting the outcome whichever way it falls.
Why not just train one shared flow on all tasks with a replay buffer? Replay is often unavailable in industrial and clinical deployments for storage and privacy reasons, as G2D argues. A shared flow trained on replayed data also brings back forgetting, which our parameter isolation removes. We include a joint-training flow on all categories as an upper reference. If it beats our routed bank by a wide margin, that would show that per-task density models lose information shared across products.
Why not just route with autoencoder reconstruction error on frozen features, which is cheaper than a flow? SurpriseNet's collapse from frozen-feature routing to pixel routing shows that reconstruction errors are fragile when compared across task models. However, it does not show that frozen-feature autoencoders fail on industrial data. Our second ablation answers this directly. If reconstruction routing matches flow routing at lower cost, we will adopt it and keep the flows only for anomaly scoring.
Decisive Checks
The experiments use the 15-category MVTec AD stream and the 12-category VisA stream in UCAD's task order. We report image AUROC, pixel AUROC, AUPRO, the forgetting measure, routing accuracy split by normal and defective images, throughput in frames per second, and memory.
The proposal stands or falls on three outcomes. The first is whether untrimmed likelihood routing misroutes defective images at a measurably higher rate than normal images, and whether trimming closes that gap. Without the first half, the core mechanism has no problem to solve. The second is whether routing on a few percent of patches keeps routing accuracy within noise of full-image routing while raising throughput roughly in proportion to $P/|S|$. That proportion is expected because routing cost scales with the number of patches each expert scores. The third, in the task-free variant, is whether the fraction of typical patches spawns new tasks for unseen categories without falsely spawning on defective images of known ones. We will compare it against a mean-likelihood threshold to check whether the likelihood paradox reappears there. If all three hold and the system beats UCAD's 93.0% on MVTec AD while staying close to the oracle, the case for one density model serving as both router and detector is made.
References
- CFLOW-AD: Real-Time Unsupervised Anomaly Detection with Localization via Conditional Normalizing Flows
- Expert Gate: Lifelong Learning with a Network of Experts
- Look At Me, No Replay! SurpriseNet: Anomaly Detection Inspired Class Incremental Learning
- Task-agnostic Continual Learning with Hybrid Probabilistic Models
- A Geometric Explanation of the Likelihood OOD Detection Paradox
- Distilling Normalizing Flows
- AE-FLOW: Autoencoders with Normalizing Flows for Medical Images Anomaly Detection
- Asymmetric Student-Teacher Networks for Industrial Anomaly Detection
- Unsupervised Continual Anomaly Detection with Contrastively-Learned Prompt
- Expert Routing with Synthetic Data for Domain Incremental Learning