Adaptive-Window Calibration of Reconstruction Anomaly Scores under Drift

A reconstruction-based anomaly detector learns to redraw normal data, typically with an autoencoder or diffusion model. It flags an input when the redrawing fails badly, meaning the reconstruction error is large. "Badly" is always relative to a reference: how large normal errors usually are. In a live stream, that reference moves. Lighting changes, sensors wear, and operating regimes switch, so the typical error on perfectly normal data rises and falls. Current streaming systems handle this in two ways. They retrain the whole generative model when a drift detector fires, or they compare each error to a rolling "mean plus one standard deviation" over a window whose length is hand-picked. Both choices sit on the same dilemma. A long history averages away noise but goes stale after a shift. A short history stays current but is noisy.

We propose to move the temporal adaptivity out of the expensive generative model and into the cheap scoring layer. We would then choose the scoring layer's memory length with a provable adaptive-window rule. The key observation makes this possible. "Is this residual above threshold τ?" is a class of one-dimensional threshold functions with VC dimension 1, and evaluating it needs no labels. This means a label-dependent adaptive-window algorithm from supervised learning applies directly to unsupervised anomaly scoring. That algorithm doubles the window until the recent and older halves disagree by more than sampling noise. The result is a threshold, and an optionally locally conditioned error baseline, that tracks drift at close to the rate an oracle knowing the drift magnitude would achieve, with no tuning of the window length.

Core Claim

Holding a reconstruction model fixed, or retraining it rarely, and recalibrating its error distribution with a doubling-window discrepancy test should do two things. It should keep the false-alarm rate close to a target level across abrupt, gradual, and recurrent drift. It should also match or beat detectors that retrain window-diverse model ensembles, at a small fraction of their compute. The claim fails if a fixed window or an exponential moving average, tuned per stream, does equally well across mixed drift schedules. That comparison is the central test.

Where this sits in prior work

The method comes from An Adaptive Algorithm for Learning with Unknown Distribution Drift. That paper expands a window as r, 2r, 4r and so on. It stops once the discrepancy between the newest r samples and the preceding r samples exceeds statistical error. It proves performance within a constant factor of a drift-aware oracle, and it recovers i.i.d. rates when there is no drift. Its stated limitation is that the discrepancy requires labels and tractable hypothesis classes, which is why it has not been used with deep generative detectors. We sidestep that limitation rather than solve it. We never apply the test to the deep network. We apply it to the scalar residual stream, where the relevant discrepancy over threshold functions is the Kolmogorov–Smirnov distance between two empirical error distributions. Its noise level follows from the Dvoretzky–Kiefer–Wolfowitz bound, roughly $\sqrt{\log(1/\delta)/r}$ for window size $r$ and failure probability $\delta$.

The analysis in On recursive estimation for time varying autoregressive processes supplies the language for the error decomposition we measure. It splits tracking error into a variance term that shrinks with memory and a drift bias that grows with memory.

Three connecting works motivate the gap:

Two works constrain the design:

Research Program

The system has three parts.

The first is a reconstruction model trained once on an initial normal segment. Depending on the domain, this is a convolutional autoencoder with a combined squared-error and structural-similarity loss, a DRÆM-style reconstructive network, or a variational autoencoder for tabular streams.

The second is a global calibrator. At each step it runs the doubling-window test on the stream of recent reconstruction errors. It then sets the alarm threshold at the $(1-\alpha)$ empirical quantile of the selected window, where $\alpha$ is the target false-alarm rate.

The third is a local variant. It clusters training latents and runs a separate adaptive window per cluster. Each error is then scored relative to its cluster's current median, which amounts to ARES made temporal. The local variant faces a sharper tradeoff, because each cluster receives fewer samples. Its adaptive windows should therefore come out longer, and we will measure whether the gain from local heteroscedasticity outweighs the added variance.

Two further mechanisms address known weaknesses. The first is contamination by anomalies, the distinction raised by VAE++ESDD's authors. Because anomalies are rare, they perturb the quantile of the window only by roughly the anomaly rate ε. Genuine drift instead shifts the whole error distribution, and that is what the Kolmogorov–Smirnov test detects. We will state and check the resulting condition: the calibrator is meaningful only when the target rate α is well above ε. This condition becomes binding at the 0.1% anomaly rate, and we will report that regime honestly.

The second is recurrence. We keep a small bank of past calibration snapshots, each summarized by its empirical error distribution. When the current short window matches a stored snapshot within the noise level, the calibrator reuses that snapshot's longer history. This recovers old operating modes at negligible cost instead of relearning them.

The model itself is retrained only when the selected window stays below a minimum size for a sustained period. That condition signals that the manifold, not just the error scale, has moved. We will log how often this happens.

Baselines

The naive reference is a static threshold. We compute it once on the training segment at the same α and never update it. This shows how much drift alone breaks a fixed detector.

The obvious practical competitors are fixed-window and exponential-moving-average calibration. These use the same frozen model with a rolling quantile, or with an exponentially weighted quantile. We will sweep the window length or decay rate over a grid. We report both the best single setting chosen on a held-out stream and the best setting chosen per stream in hindsight, as an oracle. Beating the held-out choice and landing within about 10% of the per-stream hindsight oracle, across streams that mix drift types, would support the claim.

The wavelet baseline applies Haar and DB8 soft-thresholding, following Adaptive Estimation and Learning under Temporal Distribution Shift. It runs on the per-step sequence of batch quantiles, uses the same frozen model, and gets the same tuning budget for the basis choice.

The strongest existing method on the streaming benchmark is VAE++ESDD. On MNIST-multi it reports a G-mean of 0.825, against 0.736 for StrAEm++DD, in Drift-Aware Variational Autoencoder-based Anomaly Detection with Two-level Ensembling. We will re-run it on identical streams with its published configuration. We will match its encoder architecture to ours, so that differences come from the adaptation strategy. We report G-mean, and we report wall-clock time and model-evaluations per sample as cost.

On multivariate time series the strongest reported number is D³R's SMD F1 of 0.8682, improving on 0.7571 for MTAD-GAT, from Drift doesn't Matter. For PSM and SWaT, no reported number was found on Lacuna. We will compare under the paper's affiliation-F1 protocol.

There are three ablations. The first removes the doubling test while keeping everything else, which collapses to the fixed-window baseline and isolates the window rule. The second removes local conditioning. The third removes the snapshot bank, which isolates recovery from recurrent drift.

Why not just…?

Why not just use a fixed sliding window or an exponential moving average for the threshold? This is the real threat. Under a single steady drift rate, a well-tuned fixed window is close to optimal, and our method can at best match it. The argument for adaptivity is that no single window suits a stream that alternates between stationary periods and abrupt jumps. The supporting evidence so far comes from adjacent settings. Wavelet and adaptive estimators beat fixed sliding averages on Doppler and piecewise-constant signals in Adaptive Estimation and Learning under Temporal Distribution Shift, and the oracle inequality of An Adaptive Algorithm for Learning with Unknown Distribution Drift formalizes the same point. Our mixed-schedule streams settle it directly. If the held-out-tuned fixed window matches us there, the window rule adds nothing, and we would report that.

Why not just retrain the reconstruction model on recent data, as VAE++ESDD does? Retraining adapts the manifold, which is sometimes necessary. It costs a full training run, however, and it discards recurring modes. Our hypothesis is that most drift in these benchmarks rescales errors without moving the manifold, so recalibration recovers most of the benefit. The head-to-head comparison, together with the logged count of triggered retrains, measures how much of VAE++ESDD's G-mean comes from retraining that recalibration cannot replace.

Why not just smooth the error sequence with wavelets? Wavelet thresholding is a legitimate drift-adaptive estimator and may be competitive. It assumes sub-Gaussian i.i.d. noise and needs a basis chosen by tuning. Reconstruction errors are skewed, and we care about an upper quantile, not a mean, so we expect the distribution-level Kolmogorov–Smirnov test to be better suited. Running the wavelet baseline under an equal budget answers this empirically.

Why not just use a memory-bank detector such as PatchCore, which needs no reconstruction? PatchCore was the most robust model on Robust AD, yet that benchmark still saw AUROC fall as low as 10–20% under some PCB lighting shifts. Its nearest-neighbour distances are also a scalar score whose normal distribution drifts. Our calibrator is therefore complementary, not competing, and we will apply it to PatchCore distances as well.

Proposed Experiments

Controlled synthetic streams. We build streams from MNIST-multi and synthetic generators in the style of VAE++ESDD, with abrupt, incremental, recurrent, and mixed drift at 1% and 0.1% anomaly rates. On these we plot the realized false-alarm rate over time against α, the detection delay after each shift, and G-mean. The decisive check is whether the adaptive window's excess error relative to the best per-stream fixed window stays bounded across all schedules, while every individual fixed window fails on at least one schedule.

Multivariate time series. On PSM, SMD, and SWaT we wrap a lightweight autoencoder with the calibrator. The question is whether it approaches D³R's 0.8682 on SMD at a fraction of D³R's roughly 109 MB footprint. If it falls clearly short, we conclude that structural decomposition matters there and say so.

Image streams under physical drift. We order RobustAD images by shift factor, such as gradually changing lighting or pose, to form a drift stream. We then compare DRÆM-style and PatchCore scores, each with and without calibration. AUROC computed on time-normalized scores, sPRO is not applicable, and false-alarm tracking are the metrics.

Contamination and recurrence. We sweep the anomaly rate ε toward α to locate where calibration breaks down, as the analysis predicts. Separately, we replay earlier regimes to measure how quickly the snapshot bank restores calibration, compared with resetting the history.