Calibrated Anomaly Thresholds from Exogenous-Scaled, Blocked Residuals

Streaming anomaly detectors usually turn a prediction error into an alarm with a rule like "flag if the error is more than $k$ recent standard deviations away." That rule quietly assumes three things. It assumes recent errors are independent. It assumes the recent window contains no anomalies. It assumes the only reason the noise level changes is the sensor itself. All three are false in physical systems. Errors are autocorrelated, anomalies arrive in bursts that inflate the recent spread and hide themselves, and much of the change in noise level is driven by outside factors such as rain, holidays or control commands. In practice the stated false-alarm rate of such detectors, meaning the fraction of normal points the rule is supposed to flag, is not what they actually deliver. That matters because An Evaluation of Anomaly Detection and Diagnosis in Multivariate Time Series found that the scoring and thresholding step often matters more than the model architecture.

We propose a single mechanism that repairs all three assumptions together. The noise scale is modeled as a known function of exogenous covariates (outside variables like weather or schedules) times a slowly drifting multiplier. The multiplier is estimated online as the median across interlaced blocks of recent errors, with buffer gaps between the blocks. The gaps make blocks nearly independent, so the estimate comes with honest uncertainty. The median makes the estimate ignore any burst that touches fewer than half the blocks. The exogenous factor keeps weather-driven variance swings from registering as faults. The deliverable is a threshold whose realized false-alarm rate tracks a user-chosen target on dependent, context-driven streams. It should also stay sensitive during sustained anomalies.

Core Claim

The claim is that one data structure, interlaced blocks separated by gaps longer than the dependence time, can serve two roles at once. The first is decorrelation, which gives valid concentration of the scale estimate. The second is contamination robustness, which the median over blocks provides. We further claim that conditioning the scale, not only the mean, on exogenous variables is what separates benign context-driven variance shifts from faults.

We make one concrete prediction. At a nominal false-alarm rate $\alpha$, the proposed threshold's realized rate on normal segments stays within a small factor of $\alpha$. Sliding-window Gaussian scoring overshoots on autocorrelated streams and undershoots during bursts. The proposed threshold also keeps detecting throughout anomaly bursts shorter than roughly half the estimation window.

The closest existing pieces each supply one part. The empirical Bernstein work of Mirzaei et al. partitions a trajectory into $2m$ blocks with mixing-time gaps and uses empirical variance surrogates to obtain data-dependent bounds. It is an offline tool for operator learning, and we reuse its blocking technique in a streaming, robust form. Fast Gaussian Process Approximations for Autocorrelated Data chooses a thinning interval from the partial autocorrelation function (PACF), which measures how much each lag still predicts the present after accounting for shorter lags. It shows that estimating hyperparameters on thinned blocks improves RMSE by 16% on autocorrelated SCADA data. It does batch regression only, and we borrow its PACF rule for choosing the gap. Neither paper thresholds anything, tracks scale online, or uses exogenous covariates in the noise model.

Research Program

Exogenous scale model. A forecaster conditioned on exogenous inputs $u_t$ produces residuals $e_t$. We use a dedicated external sub-network fused with the endogenous branch, following ST-ResNet. That paper found naive concatenation of external metadata inferior; its dedicated external branch reached TaxiBJ RMSE 16.89, against 18.18 for DeepST.

We then model the conditional noise scale as

$$\sigma_t^2 = s_t \, h_\theta(u_t),$$

where $h_\theta$ is a small network mapping exogenous covariates to a relative variance and $s_t$ is a slowly drifting multiplier that captures sensor aging and other unexplained drift. We fit $h_\theta$ on normal training data by maximizing a Gaussian or Student-t likelihood of $e_t$ and hold it fixed at test time. Only $s_t$ is tracked online, so a storm that is visible in the covariates raises $\sigma_t$ without being treated as an alarm.

Blocked median scale tracking. Over the most recent window of length $W$, the standardized squared errors $z_t = e_t^2 / h_\theta(u_t)$ are split into $2m$ consecutive blocks of length $\tau$, and only the odd-numbered blocks are kept. Each kept block $k$ yields its mean $v_k$, and

$$\hat s_t = \operatorname{median}(v_1,\dots,v_m).$$

The gap $\tau$ is set from the PACF of $z_t$, which is the squared residual rather than the residual. This matters because the dependence that corrupts scale estimation is clustering of large errors, which linear whitening does not remove. We re-estimate $\tau$ on a slow schedule.

Standard blocking arguments give two bounds for a β-mixing process, meaning one whose dependence between distant points fades as the gap grows. First, the kept blocks can be coupled to independent copies at a cost of about $m\beta(\tau)$, where $\beta(\tau)$ is the mixing coefficient at lag $\tau$. Second, a median of $m$ near-independent block means deviates from its target with probability decaying exponentially in $m$. Together these give a confidence band $[\hat s^-_t, \hat s^+_t]$. A burst of length $L$ corrupts about $L/(2\tau)+1$ kept blocks, so the median is protected while $L \lesssim m\tau = W/2$. We will state this result carefully and check it empirically. We will not assume it holds.

Threshold. The alarm fires when $e_t^2 > \hat s^+_t \, h_\theta(u_t)\, q_\alpha$. Here $q_\alpha$ is the upper-$\alpha$ quantile of the standardized error $e_t^2/\sigma_t^2$. It is estimated on training data, either by an exponential tail fit applied to block maxima or by an empirical quantile. Using block maxima avoids the clustered-exceedance bias that hurts EVT (extreme value theory) tail fits on dependent scores. EVT tail fitting of this kind was used on river sensors by Talagala et al., who assume independent tails. Using the upper confidence limit $\hat s^+_t$ rather than the point estimate $\hat s_t$ is what turns a heuristic into a rate guarantee. The threshold widens exactly when the effective sample size is small.

A secondary use is to supply the variance proxy that the Anytime Tracking CUSUM change detector of Gafni, Iyengar and Zeevi assumes is known. That detector obtains near-optimal dynamic regret, the extra error accumulated from tracking a signal that shifts, of order $O(\sigma^2(S+1)\log T)$ over $S$ change points and horizon $T$. Its authors leave online variance calibration open. We will test whether plugging in $\hat s^+_t h_\theta(u_t)$ preserves its behavior on the Numenta Anomaly Benchmark (NAB).

Baselines

The naive reference is a static threshold. We fit a Gaussian or an exponential tail to training errors once and never update it. This is the practice that Orchard et al. and AlerTiger describe as failing under drift. It sets the floor.

The strongest existing competitor is the Univariate AutoEncoder with dynamic Gaussian scoring (Gauss-D) from Garg et al.. Gauss-D standardizes each error by the mean and variance of a sliding test window, and this pairing achieved the highest average rank across SWaT, SMD and MSL. No numeric value survives in the file: no reported result found on Lacuna for UAE + Gauss-D on SWaT, SMD or MSL. We will rerun it from the authors' protocol. For fairness, every threshold method will score the same residual stream from the same forecaster, trained once. Window length and $k$ get the same tuning budget: a grid of equal size selected on a validation split by composite F-score. Beating it means higher composite F-score $Fc_1$ on at least two of three datasets and realized false-alarm rate closer to nominal on normal segments.

The strongest simple robust competitor is a rolling median absolute deviation (MAD) or interquartile-range scale in the style of AlerTiger, which widens bands during benign variance spikes. We will also run it with our exogenous factor $h_\theta$ added. That variant matches our contamination robustness on paper, so beating it isolates what blocking and the confidence band contribute. On the river data the domain reference is Talagala et al.'s pipeline, which combines one-sided derivatives, k-nearest-neighbor distance scores and an EVT threshold. It reports Optimized Precision (OP) of 0.8329 on Sandy Creek and 0.8845 on Pioneer River, and we compare on OP to match their reporting. On Kitsune the reference is Griffin, whose threshold tracks a moving average of recent normal reconstruction errors and which reports MCC 0.982. That result is essentially saturated, so we use Kitsune only for a slow-poisoning stress test, not a headline number.

Three ablations each remove one ingredient. Blocks without gaps tests decorrelation. Mean-of-blocks instead of median tests robustness. Setting $h_\theta \equiv 1$ with the exogenous variables kept in the forecaster's mean isolates the claim that exogenous conditioning of the scale, not only the mean, matters.

Why not just…?

Why not just prewhiten the residuals with an autoregressive model and threshold the innovations? This is the most serious alternative. An autoregressive fit removes linear autocorrelation of $e_t$, but it does not remove clustering in $e_t^2$. It also still needs a scale estimate that anomaly bursts will contaminate. We include "autoregressive prewhitening plus rolling MAD plus $h_\theta$" as a baseline. If it matches our realized false-alarm calibration on the real datasets, the blocking ingredient adds nothing and we will say so. The contribution would then reduce to exogenous scale conditioning. The synthetic study below is designed to show where the two diverge: streams with volatility clustering but little linear autocorrelation.

Why not just use sliding-window Gauss-D, which already wins benchmark rankings? Its ranking in Garg et al. was obtained without exogenous inputs and without measuring realized false-alarm rates. Its window mean and variance absorb sustained anomalies and treat correlated errors as independent. Our burst-length sweep measures directly when Gauss-D stops detecting inside a burst, and the calibration plots measure how far its nominal rate is from its realized one.

Why not just put the exogenous variables into the forecaster and leave thresholding alone? Conditioning the mean removes predictable level shifts. It does not handle the case where rain makes turbidity noisier as well as higher. The $h_\theta\equiv1$ ablation answers this. If it performs as well on the river and injected-TaxiBJ data, the scale model is unnecessary.

Why not just use robust rolling statistics, which tolerate contamination too? A rolling MAD does tolerate bursts up to half its window. We concede this, which is why it is a main baseline and not a straw man. What it lacks is a sampling-uncertainty correction under dependence, so its realized false-alarm rate should drift from nominal as autocorrelation increases. The synthetic mixing-time sweep tests exactly this.

Proposed Experiments

The decisive synthetic study generates Ornstein–Uhlenbeck processes with tunable mixing rate, following Mirzaei et al.. We multiply them by a volatility process that is partly driven by a simulated exogenous covariate and partly by hidden persistent (GARCH-like) volatility. We then inject anomaly bursts of controlled length $L$. We sweep mixing time, the share of variance explained by the exogenous covariate, and $L/W$. For each setting we plot realized versus nominal false-alarm rate on normal segments and within-burst recall as a function of $L$.

The method passes if three things hold. Its realized rate stays within a factor of two of $\alpha$ across mixing times, where Gauss-D and rolling MAD are expected to drift. Its recall stays high up to $L \approx W/2$ and drops after. The exogenous-scale variant cuts false alarms during covariate-driven variance surges relative to $h_\theta\equiv1$.

On real data we use four sources. SWaT, SMD and MSL are scored by $Fc_1$, the composite F-score from Garg et al. that combines time-wise precision with event-wise recall. We avoid point-adjusted F1 because Garg et al. showed it rewards trivial detectors. On SWaT, actuator commands serve as exogenous inputs. The Sandy Creek and Pioneer River sensors are scored by OP. There, river level and, where available, rainfall serve as exogenous drivers of turbidity and conductivity, which tests separation of technical faults from real hydrological events. TaxiBJ and BikeNYC contribute their weather and holiday covariates, with anomalies injected, since they carry no labels. NAB AWS CPU utilization tests the plug-in variance for change detection.

Across all of these we report realized false-alarm rate on labeled-normal segments alongside detection metrics. Calibration is the property the method claims. If our method wins on $Fc_1$ but is no better calibrated than rolling MAD with exogenous scaling, the central claim fails.