Anomaly detectors that watch live data, such as network traffic, sensor feeds, or camera streams, must keep updating their idea of "normal," because normal changes over time. That need to update is also the weak point. An attacker can feed the detector a slow trickle of near-normal inputs until its idea of normal includes the attack. This is called slow-burn poisoning. The defender appears to face two bad options. Updating freely lets the attacker steer the detector. Refusing to update produces floods of false alarms on legitimately changed data, which is the streaming version of the familiar loss of clean accuracy that robustness defenses cause. Our proposal rests on one observation about where the data comes from. Real change in a deployment, such as a firmware rollout, a day-night cycle, or seasonal load, usually shows up across many independent sources at once. A poisoning campaign comes from the few sources the attacker controls.
We propose that an online detector should only fold a change into its shared baseline when that change is corroborated by a majority of independent sources, such as hosts, devices, or cameras. Changes seen only in a few sources go into small per-source adjustments with a capped budget. When a source exceeds its budget, the system raises a visible review alarm instead of silently accepting the change. This turns the poisoning question from an unsolvable per-sample guess ("is this point drift or attack?") into a quantity we can measure: how fast an attacker holding a fraction of sources can move the baseline. The clean cost also becomes measurable, as review alarms raised on sources that genuinely changed alone.
Where the field stands
Online detectors handle the drift-versus-outlier question one stream at a time.
- FITNESS scans backward for the longest window of statistically similar samples. It proves sub-linear regret for random, sub-Gaussian outliers, but not for an attacker who keeps each point inside the similarity test.
- MIDAS-F skips baseline updates whenever a sample scores above a fixed threshold. An attacker can stay just below that threshold.
- RRBM-DD softly truncates each update to limit the influence of any single sample. A slow-burn attack, however, works through the sum of many small, individually unremarkable updates.
These three papers supply the baselines we must beat. They share a blind spot. Each treats the stream as one undifferentiated sequence, so a sustained, well-camouflaged shift is statistically identical to real drift.
Two results motivate the source-structure idea. Can't Boil This Frog attacked online autoencoders on the SWaT water-treatment testbed. It found that cross-sensor physical relationships made poisoning hard: only 4 of 10 relevant attacks succeeded, and only after their magnitudes were clipped. We read this as evidence that redundancy an attacker does not control is what protects an online learner. Our proposal is an attempt to manufacture such redundancy in streams that have no physical laws, such as network traffic and images, by using independence across sources instead.
The adversarial out-of-distribution literature constrains the threat model. A detector of out-of-distribution (OOD) inputs flags data unlike the training distribution. That literature shows attackers can shape inputs so that their internal representations match normal data:
- Liu et al. and Yao et al. shape inputs to mimic normal internal representations.
- EvG collapses detectors with low-dimensional transforms that evade perturbation-hardened defenses.
So any defense that relies on per-sample anomaly scores must assume those scores can be forged. Robust OOD detectors such as ALOE, AROS, and AOE harden a static detector. None of them say how to move that detector's boundary safely when normal data drifts. g-LOND controls false discoveries in streaming OOD detection, but only against a fixed calibration set that becomes stale under drift.
On the cost side, BaDLoss shows what aggressive filtering costs. It removes up to 40% of training data, and CIFAR-10 clean accuracy drops from about 91.7% to about 88.4%, with minority classes hit hardest. Our corroboration rule will have its own version of that tail harm: sources that legitimately drift alone. We treat this as the primary cost metric rather than hiding it.
Core Claim
Suppose an attacker controls fewer than half of $K$ independent sources. Then median-based corroboration caps how fast the shared baseline can be dragged per update. The cap is set by the spread of the honest sources, not by how hard the attacker optimizes. Concretely, let $\Delta_k$ be the update proposed by source $k$'s recent window, for example the shift in feature means or the gradient of the detector's loss. Let $\hat\Delta$ be their coordinate-wise median, and let $\rho$ be the fraction of sources the attacker controls. Then the per-step bias the attacker can induce satisfies
$$ |\hat\Delta - \Delta^{\text{honest}}_{1/2}| \;\le\; \Delta^{\text{honest}}_{1/2+\rho} - \Delta^{\text{honest}}_{1/2-\rho}, $$
where $\Delta^{\text{honest}}_{q}$ is the $q$-quantile of the honest sources' proposals. This bound follows from the standard breakdown property of the median.
The bound does not stop slow-burn poisoning. It turns it into a rate limit. To move the baseline by a distance $D$ toward a target attack, the attacker needs at least $D$ divided by that quantile gap in steps. This gives a falsifiable prediction: attack time-to-success should grow as honest sources become more homogeneous, and as $\rho$ shrinks.
Change confined to one source accumulates in that source's own adjustment. Once the adjustment exceeds a budget $B$, the system raises a review alarm. A compromised host can therefore blind detection only on itself, and only by tripping a visible alarm.
Research Program
We will build the corroborated updater as a wrapper around existing online detectors. The wrapped detectors are Kitsune's autoencoder ensemble (Kitsune) and the FITNESS context selector for network streams, plus an energy-based OOD detector in the style of AOE for image streams. The wrapper has three parts:
- Corroboration step: form per-source proposals and update the shared baseline with their clipped median.
- Local adjustments: keep a per-source residual whose size is capped by $B$.
- Escalation rule: raise a review alarm when a source's residual exceeds $B$.
The wrapper adds work linear in the number of sources per update window, which avoids the 30–60× overhead of back-gradient methods.
Equally important is the attacker, because no adaptive drift-mimicking attack exists against these detectors. We will implement two attacks. The first is interpolative poisoning, which feeds inputs that move a little closer to the target attack at each step. The second is back-gradient bilevel poisoning, which differentiates through the detector's training updates. Both follow Can't Boil This Frog, extended so the attacker knows the defense:
- Against FITNESS, the attacker keeps every point inside the similarity test.
- Against MIDAS-F, it keeps every point below the rejection threshold.
- Against our wrapper, it places compromised sources' proposals at the edge of the honest spread.
For images, poisoned inputs will use feature-matching perturbations so their anomaly scores look normal.
Baselines
The naive reference points are a frozen detector that never updates and an always-update detector that trusts every sample. The frozen detector shows the clean cost of refusing to adapt, measured as false alarms after benign drift. The always-update detector shows the undefended attack success rate. A fixed sliding window, tuned on a held-out drift segment, adds the simplest middle ground.
The strongest stream-level competitor is FITNESS. It directly addresses drift versus outliers and reports an Average Precision Score of 0.96 on the Kitsune IoT attack data (FITNESS). We will run it on identical streams with its noise-variance estimate tuned using the same budget we give our own budget $B$. Beating it means two things together:
- a lower adaptive-attack success rate, or a longer attack time-to-success, at matched poison budget;
- clean average precision under benign drift within one point of FITNESS's.
MIDAS-F is the strongest conditional-update baseline on graph streams. It reports up to 62% ROC-AUC improvement over baselines on CTU-13 (MIDAS-F). We will tune its threshold on the same benign-drift segment and compare under the same adaptive attacker. RRBM-DD (RRBM-DD) reports an average rank of 1.00 on its 12 MOA streams, but the file gives no absolute numbers. We will rerun it where the stream format allows. For image streams the reference is g-LOND, which reports FPR falling from 45.63% to 30.62% on Places365 with CIFAR-100 as normal data (g-LOND). We will run it with a static calibration set, and also with a naively refreshed one, under the same drift.
The key ablation replaces source-based grouping with random grouping of the same samples into $K$ pseudo-sources, keeping the median and budget machinery identical. If random groups defend as well as real sources, then source independence is not doing the work and the core claim fails.
Why not just…?
Why not just reject updates whose anomaly score exceeds a threshold, as MIDAS-F does? A threshold only constrains individual points. Feature-camouflage attacks (Liu et al.) and drift-mimicking sequences can keep every point below it. Our adaptive-attacker experiments on CTU-13 test exactly this. We expect MIDAS-F's attack time-to-success to stay roughly constant as the attacker adapts, while ours scales with honest spread. If MIDAS-F holds up just as well, the added machinery is unjustified and we will say so.
Why not just use FITNESS's adaptive window? FITNESS's guarantee assumes random sub-Gaussian outliers. An attacker who moves the running mean in steps smaller than the $1/\sqrt{j}$ acceptance threshold, where $j$ is the window length, is treated as legitimate drift. FITNESS may even shorten its window and adapt faster toward the attack. This is a real competitor on benign drift, so we report its clean average precision alongside ours. Our claim is limited to the adaptive-attack regime.
Why not just freeze the detector and retrain periodically on verified data? Freezing produces the persistent false-alarm cascades documented by Wulfmeier et al.. Verified retraining needs a clean label or anchor source, which BaDLoss-style pipelines assume (BaDLoss) but streams rarely have. The frozen baseline and a "retrain every $N$ hours on signature-filtered data" variant quantify that cost directly.
Why not just bound each sample's influence, as RRBM-DD does? Per-sample truncation limits how much any one point moves the model. Slow-burn poisoning succeeds through the accumulated total of many small updates. Comparing cumulative baseline displacement under matched poison budgets will show whether truncation alone rate-limits the attack. We acknowledge that our wrapper is a form of robust aggregation. The new parts are applying it to the decision of whether to accept drift, the per-source escalation budget, and measuring the clean cost it creates.
Proposed Experiments
We will use four testbeds:
- Kitsune network captures: split by device so each IoT host is a source.
- CTU-13 graph streams: split by host.
- SWaT: sources are subsystem sensor groups. This is a control setting where physical invariants already help.
- CIFAR-10 image stream: sources are ten simulated cameras, benign drift is a gradual increase in corruption severity applied to all cameras, and some cameras receive camouflaged OOD inputs drawn from CIFAR-100.
On security streams, benign drift is injected both globally (all sources) and locally (one source) to separate the two clean-cost regimes. Splitting the Kitsune and CTU-13 captures by host is our own preprocessing and must be validated: if hosts are too few or too heterogeneous, the honest spread will be wide and the bound weak, and we will report this.
The metrics fall into three groups:
- Attack: success rate and time-to-success.
- Clean, global drift: false-alarm rate and adaptation lag, meaning time until false alarms return to their pre-drift level.
- Clean, local drift: review-alarm rate on legitimately drifting sources, plus image classification accuracy on clean drifted data.
Three checks decide the outcome:
- Time-to-success must grow as the honest quantile gap shrinks, consistent with the bound. We will vary the number of compromised sources and how homogeneous the honest sources are.
- Real-source grouping must clearly beat random grouping.
- Under global benign drift, adaptation lag must stay within that of FITNESS. If corroboration only buys robustness by refusing global drift, we have recreated the robustness-accuracy trade-off rather than escaped it.
References
- FITNESS: (Fine Tune on New and Similar Samples) to detect anomalies in streams with drift and outliers
- Real-Time Anomaly Detection in Edge Streams
- Adversarial concept drift detection under poisoning attacks for robust data stream mining
- Can't Boil This Frog: Robustness of Online-Trained Autoencoder-Based Anomaly Detectors to Adversarial Poisoning Attacks
- Manipulating Machine Learning: Poisoning Attacks and Countermeasures for Regression Learning
- Adversarial Medical Image With Hierarchical Feature Hiding
- Adversarial Distributions Against Out-of-Distribution Detectors
- Robust Out-of-distribution Detection for Neural Networks
- Adversarially Robust Out-of-Distribution Detection Using Lyapunov-Stabilized Embeddings
- Out-of-Distribution Detection with Adversarial Outlier Exposure
- An Online Statistical Framework for Out-of-Distribution Detection
- Protecting against simultaneous data poisoning attacks
- Kitsune: An Ensemble of Autoencoders for Online Network Intrusion Detection
- Incremental Adversarial Domain Adaptation for Continually Changing Environments