Poisoning Attacks on Adaptive Anomaly Detection Baselines

This direction is split into other directions: online learning vulnerability. Its last version stays readable and citable here.

Version 1 of 152 papers67 conceptsfrozen 2026-10-04

In modern cybersecurity, the definition of "normal" is rarely static. Network traffic patterns shift with the time of day, user behaviors evolve, and industrial processes undergo natural drift. To keep pace, security systems have moved away from rigid, rule-based signatures toward adaptive, unsupervised machine learning models—such as autoencoders and clustering algorithms—that continuously update their internal representation of normality based on live data streams.

However, this adaptability creates a profound security paradox. If a system learns what is "normal" by observing the environment, an adversary can manipulate that environment to redefine normality itself. This research direction explores adversarial baseline poisoning, a class of attacks where malicious actors slowly inject subtle deviations into training data to "blind" detection systems to future attacks.

The Boiling Frog Strategy

The core intuition behind baseline poisoning is often described as "boiling the frog." If an attacker launches a massive, sudden spike in traffic, a forecasting algorithm like the one proposed by Huy Hoàng Nguyễn and colleagues will immediately flag it as an anomaly. But if the attacker introduces the malicious pattern incrementally, the adaptive model may incorporate these small deviations into its baseline.

As Moshe Kravchik demonstrates, online-trained autoencoders are particularly susceptible to this gradual manipulation. By exploiting the update mechanism intended to handle natural concept drift, an adversary can shift the model's "normal" manifold. Over time, the statistical threshold for what constitutes an alert moves, eventually allowing significant malicious activity to be misclassified as benign behavior.

The Feedback Loop Vulnerability

The vulnerability stems from a fundamental feedback loop in online learning. Systems like Griffin, developed by Liyan Yang and Bin Xiao 0001, update their models with every new packet to maintain high accuracy in dynamic Software-Defined Networks. While this prevents the model from becoming obsolete, it creates an opening for "accumulative poisoning."

Jianing Zhu and Xiawei Guo highlight that these modifications are individually imperceptible but collectively degrade model performance. This is especially dangerous for detecting Advanced Persistent Threats (APTs). As Fan Xu notes in the development of LTRDetector, if an attacker is already present in the system when the monitoring tool is deployed, the "clean" baseline is poisoned from day one. The model effectively learns to ignore the very threat it was built to find.

The Drift versus Outlier Dilemma

A central challenge for these systems is distinguishing between concept drift (a legitimate change in the environment) and adversarial poisoning (a malicious attempt to shift the model). Jingjing Fei points out that incorrectly updating a model with anomalous data leads to model poisoning, while failing to update during legitimate drift leads to a flood of false alarms.

To address this, Abishek Sankararaman and Balakrishnan Narayanaswamy proposed the FITNESS framework, which uses a mathematically rigorous approach to fine-tune models only on recent samples that are "similar enough" to the existing distribution. This attempts to create a "buffer" that accepts gradual change while rejecting the sharp, localized perturbations typical of many attacks.

Robustness and Defensive Architectures

Researchers are increasingly focusing on making the underlying architectures of these detectors more resilient to contamination.

  1. Memory-Based Filtering: Siddharth Bhatia and Bryan Hooi developed MemStream, which uses a FIFO-updated memory module to store "normal" patterns. By comparing incoming data against this explicit memory, the system can resist "memory poisoning" even as it adapts to new streams.
  2. Ensemble Filtering: Xiangyu Xi and Shikun Zhang suggest using "strictly filtered" ensembles. By combining multiple models (like One-Class SVMs and Isolation Forests), the system can achieve high recall even if the training data contains undetected anomalies.
  3. Collaborative Consensus: In the IoT space, Tomer Golomb and Yisroel Mirsky introduced CIoTA, which uses blockchain to allow multiple devices to collaboratively train a model. This makes it harder for an attacker to poison the baseline, as they would need to compromise a majority of the participating devices to shift the global consensus of what is "normal."
  4. Distributional Quarantine: Zhen Xiang and David J. Miller 0001 proposed using mixture models to isolate suspicious training data into separate "quarantine" components, preventing malicious samples from corrupting the primary model of benign behavior.

Conclusion

The shift toward adaptive anomaly detection is a necessity in a world of shifting data distributions. However, as Guido Schwenk and Sebastian Bach early identified, the very mechanism that allows these systems to learn also allows them to be deceived. Understanding the dynamics of baseline poisoning is critical for building "assured AI" that can distinguish between a system that is naturally evolving and one that is being slowly led astray.

Papers in this version

History & citationVersion 1 · 2026-10-04

Split into other directions · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_poisoning_attacks_on_adaptive_anomaly_de_b699368a,
  title = {Poisoning Attacks on Adaptive Anomaly Detection Baselines},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/poisoning-attacks-on-adaptive-anomaly-detection-baselines/rdr_b699368a88dd47b39caa059a3a76fdef},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_poisoning_attacks_on_adaptive_anomaly_de_b699368a_v1,
  title = {Poisoning Attacks on Adaptive Anomaly Detection Baselines},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/poisoning-attacks-on-adaptive-anomaly-detection-baselines/rdr_b699368a88dd47b39caa059a3a76fdef/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}