Perturbation Learning Based Anomaly Detection

Jinyu Cai, Jicong Fan

Abstract

This paper presents a simple yet effective method for anomaly detection. The main idea is to learn small perturbations to perturb normal data and learn a classifier to classify the normal data and the perturbed data into two different classes. The perturbator and classifier are jointly learned using deep neural networks. Importantly, the perturbations should be as small as possible but the classifier is still able to recognize the perturbed data from unperturbed data. Therefore, the perturbed data are regarded as abnormal data and the classifier provides a decision boundary between the normal data and abnormal data, although the training data do not include any abnormal data. Compared with the state-of-the-art of anomaly detection, our method does not require any assumption about the shape (e.g. hypersphere) of the decision boundary and has fewer hyper-parameters to determine. Empirical studies on benchmark datasets verify the effectiveness and superiority of our method.

Introduction

Anomaly detection (AD) is a critical task in machine learning, often described as finding a "needle in a haystack." The primary challenge lies in the inherent imbalance of training data: models are typically exposed almost exclusively to "normal" samples. Traditional approaches, such as Support Vector Data Description (SVDD), attempt to address this by assuming a specific geometric shape for the data, such as a hypersphere. However, high-dimensional data like images rarely conform to such simple shapes, leading to poor generalization.

Perturbation Learning Based Anomaly Detection (PLAD) introduces a novel framework that eliminates these rigid assumptions. Instead of forcing the data into a predefined shape, PLAD learns to generate its own "anomalies" by applying minimal perturbations to normal data. This allows the model to learn a precise decision boundary that naturally follows the manifold of the normal data.

Methodology: The Perturbator-Classifier Framework

The core of PLAD is a joint optimization between two neural networks: a Perturbator ($g$) and a Classifier ($f$).

PLAD Architecture and Conceptual Boundary

The Perturbator

The perturbator takes a normal training sample $x$ and applies two types of transformations:

  • Multiplicative Perturbation ($\alpha$): Scales the input features.
  • Additive Perturbation ($\beta$): Shifts the input features.

The objective of the perturbator is to find the smallest possible change that transforms a normal sample into something the classifier identifies as "abnormal." By keeping these changes minimal, the system ensures that the synthetic anomalies are located just outside the boundary of the normal data distribution.

The Classifier

The classifier is trained to distinguish between the original normal data and the perturbed samples. Because the perturbations are constrained to be small, the classifier is forced to learn a very tight decision boundary that "hugs" the natural shape of the normal data.

An advanced variant of PLAD utilizes a Variational Autoencoder (VAE) logic. Rather than learning a single deterministic perturbation, the VAE-based perturbator learns a distribution of possible perturbations. This increases the robustness of the classifier, allowing it to detect a wider variety of real-world anomalies.

Experimental Setup

The authors evaluated PLAD across diverse data types to demonstrate its versatility:

  • Image Datasets: CIFAR-10 and Fashion-MNIST were used, where one class is designated as "normal" and others as "anomalies."
  • Tabular Datasets: The model was tested on medical data, specifically the Thyroid and Arrhythmia datasets.
  • Multi-class Scenarios: A challenging setup where "normal" data comprises multiple categories, requiring the model to identify outliers from a heterogeneous baseline.

Results and Discussion

PLAD consistently outperformed or matched state-of-the-art methods like OCSVM, Deep SVDD, and DROCC.

Visualization of learned representations for Fashion-MNIST classes

Performance Highlights

  • Image Recognition: On CIFAR-10, PLAD achieved an average AUC of 75.1%, surpassing the previous benchmark (IGD at 74.3%). In the "Automobile" category, it showed a significant 6% lead over competitors. On Fashion-MNIST, it reached an average AUC of 93.5%.
  • Tabular Data: The model showed exceptional performance on the Arrhythmia dataset, achieving a 71.0% F1-score, which represents a 9.2% improvement over the next best method, GOCC.
  • Robustness: In multi-class tests on CIFAR-10, PLAD outperformed Deep SVDD by 9.1%, proving that it can handle complex, multi-modal normal distributions.

The primary advantage of PLAD is its adaptive nature. Unlike SVDD, which imposes a hypersphere, PLAD allows the data to define its own boundary. Furthermore, the model is easier to tune, as it relies primarily on a single hyperparameter ($\lambda$) to control the "smallness" of perturbations, reducing the need for extensive grid searches.

Conclusion

PLAD offers a robust, backbone-agnostic solution to anomaly detection by learning to generate synthetic anomalies. By focusing on minimal perturbations, it creates highly accurate decision boundaries without geometric assumptions. While currently validated on image and tabular data, the framework's flexibility suggests future applications in time-series and graph-based anomaly detection by integrating specialized architectures like RNNs or GNNs.

Summary updated

Abstract

This paper presents a simple yet effective method for anomaly detection. The main idea is to learn small perturbations to perturb normal data and learn a classifier to classify the normal data and the perturbed data into two different classes. The perturbator and classifier are jointly learned using deep neural networks. Importantly, the perturbations should be as small as possible but the classifier is still able to recognize the perturbed data from unperturbed data. Therefore, the perturbed data are regarded as abnormal data and the classifier provides a decision boundary between the normal data and abnormal data, although the training data do not include any abnormal data. Compared with the state-of-the-art of anomaly detection, our method does not require any assumption about the shape (e.g. hypersphere) of the decision boundary and has fewer hyper-parameters to determine. Empirical studies on benchmark datasets verify the effectiveness and superiority of our method.
BibTeX
@inproceedings{cai2022perturbation,
  author = {Jinyu Cai and Jicong Fan},
  title = {{Perturbation Learning Based Anomaly Detection}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2022},
  url = {https://openreview.net/forum?id=0T299NtBAA},
}
Figure