Perturbation Learning Based Anomaly Detection

Jinyu Cai, Jicong Fan

Anomaly detection (AD) is often likened to finding a needle in a haystack, with the added difficulty that you may have never actually seen the needle before. Most traditional algorithms attempt to solve this by defining what "normal" looks like and flagging anything that falls outside that definition. However, a recent paper introduces a more proactive strategy: Perturbation Learning Based Anomaly Detection (PLAD). Instead of passively waiting for anomalies to appear, PLAD learns how to create its own "fake" anomalies to better define the boundary of normality.

At its core, PLAD is a novel framework that jointly learns a "perturbator" to generate subtle, near-normal synthetic anomalies and a classifier to detect them. By training on these "near-miss" examples, the model learns a highly precise decision boundary without relying on rigid geometric assumptions about the underlying data.

The Problem with Traditional Anomaly Detection

Traditional anomaly detection faces two major hurdles:

  1. Rigid Geometric Assumptions: Many classical methods, such as One-Class SVM (OCSVM) or Support Vector Data Description (SVDD), assume the normal data fits neatly into a specific shape, like a hypersphere. In complex, high-dimensional real-world data (such as images), this assumption frequently fails.
  2. Lack of Explicit Boundaries: Deep learning methods often rely on "reconstruction error"—training an autoencoder to rebuild normal data and assuming anything it cannot perfectly reconstruct is an anomaly. The problem is that these models merely learn to compress data; they do not explicitly learn a discriminative decision boundary.

PLAD addresses these gaps by explicitly learning a discriminative boundary through targeted synthetic data generation.

The PLAD Architecture

The core of PLAD is a two-part neural network architecture designed to play a cooperative game:

  1. The Perturbator ($g$): This component takes a normal data point and applies small multiplicative ($\alpha$) and additive ($\beta$) changes. The goal is to create a "perturbed" sample that remains as close to the original as possible while technically becoming "abnormal."
  2. The Classifier ($f$): This component is trained to distinguish between the original normal data and the newly generated perturbed data.
The PLAD architecture: a perturbator generates subtle changes, and a classifier learns to distinguish the original from the perturbed sample.

The effectiveness of PLAD lies in its joint optimization. The perturbator is penalized if the changes it makes are too large, forcing it to find the "hardest" examples—those that are just barely different from normal. Meanwhile, the classifier learns to spot these tiny discrepancies. The authors also extend this framework using a Variational Autoencoder (VAE) approach to learn a distribution of perturbations for every data point, making the model even more robust.

Key Insights and Advantages

  • Shape-Agnostic Learning: Unlike SVDD, PLAD does not require the data to be shaped like a sphere, a donut, or any specific manifold. By generating perturbations locally around every data point, it "wraps" the decision boundary tightly around the actual data distribution.
  • The Power of "Near-Misses": Training a classifier on random noise is trivial. Training on "near-miss" perturbations—data points that are 99% normal—forces the classifier to learn the most essential, discriminative features of the normal class.
  • Visualizing the Gap: Using t-SNE visualizations, the authors demonstrate that PLAD successfully pushes normal and perturbed data into distinct clusters within its internal embedding space, even when the samples look nearly identical in the original input space.
t-SNE visualization showing how PLAD separates normal training data (blue) from its perturbed counterparts (orange).

Empirical Results

The researchers evaluated PLAD across four diverse benchmark datasets, covering both computer vision (CIFAR-10, Fashion-MNIST) and tabular medical diagnostics (Thyroid, Arrhythmia). PLAD consistently outperformed state-of-the-art competitors like Deep SVDD and DROCC:

  • Image Performance: On CIFAR-10, PLAD achieved an average AUC of 75.1%, surpassing the previous best of 74.3%. On Fashion-MNIST, it reached an impressive 93.5% AUC.
  • Tabular Superiority: The results on the Arrhythmia dataset were particularly striking. PLAD achieved a 71.0% F1-score, representing a massive 9.2% improvement over the runner-up method (GOCC).
  • Multi-class Robustness: In more difficult tests where "normal" data consisted of multiple different classes, PLAD maintained a significant lead, proving its capability to handle complex, multi-modal distributions.

Limitations and Future Directions

While highly effective, the authors acknowledge that the current model is limited to additive and multiplicative perturbations ($x \odot \alpha + \beta$). This formulation might not capture every possible type of real-world anomaly. They suggest that future work should focus on learning more sophisticated, general perturbation functions to handle even more complex data types. Additionally, while the framework is theoretically applicable to time-series and graph data, those specific applications remain an exciting avenue for future exploration.

Summary updated

Abstract

This paper presents a simple yet effective method for anomaly detection. The main idea is to learn small perturbations to perturb normal data and learn a classifier to classify the normal data and the perturbed data into two different classes. The perturbator and classifier are jointly learned using deep neural networks. Importantly, the perturbations should be as small as possible but the classifier is still able to recognize the perturbed data from unperturbed data. Therefore, the perturbed data are regarded as abnormal data and the classifier provides a decision boundary between the normal data and abnormal data, although the training data do not include any abnormal data.Compared with the state-of-the-art of anomaly detection, our method does not require any assumption about the shape (e.g. hypersphere) of the decision boundary and has fewer hyper-parameters to determine. Empirical studies on benchmark datasets verify the effectiveness and superiority of our method.
BibTeX
@inproceedings{cai2022perturbation,
  author = {Jinyu Cai and Jicong Fan},
  title = {{Perturbation Learning Based Anomaly Detection}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2022},
  url = {https://openreview.net/forum?id=0T299NtBAA},
}
Figure