AnoRand: A Semi Supervised Deep Learning Anomaly Detection Method by Random Labeling

Mansour Zoubeirou A. Mayaki, Michel RiveillarXiv 2023 · OpenReview

TL;DR

This paper introduces AnoRand, a novel semi-supervised deep learning method for anomaly detection. AnoRand tackles the challenge of limited labeled data by combining a deep autoencoder with a feed-forward perceptron, using synthetically generated anomalies to train the model. It demonstrates state-of-the-art performance across numerous real-world datasets, outperforming existing unsupervised and even some supervised methods.

What Problem Does This Paper Address?

Anomaly detection, the task of identifying rare events or data points that deviate significantly from the norm, is a critical but challenging problem in machine learning. A major hurdle is the scarcity of labeled data, especially for the anomalous class, which is often the class of interest. Traditional supervised methods struggle with imbalanced datasets, while unsupervised methods rely on assumptions about anomaly distribution that may not always hold. Existing semi-supervised methods often require careful tuning or specific assumptions about the data. This paper aims to bridge this gap by proposing a robust semi-supervised approach that leverages deep learning without requiring extensive labeled data.

Methods

AnoRand is a deep learning architecture that combines two key components:

  • Noise Detection (ND) Block: This block uses a Feed-Forward Perceptron (FFP). The FFP is particularly adept at learning from imbalanced data, and in this context, it's trained to identify "noisy" or synthetically generated anomalies.
  • Autoencoder (AE) Block: This standard autoencoder learns to reconstruct normal data. Anomalies are typically identified by high reconstruction errors.

The core innovation lies in how these blocks are integrated and trained:

  1. Synthetic Anomaly Generation: A small percentage (e.g., 2%) of normal training data is randomly perturbed with noise (using Gaussian noise and SMOTE) to create synthetic anomalies.
  2. Joint Training: The FFP and autoencoder blocks are trained jointly in an end-to-end fashion. This allows the FFP to reinforce the autoencoder's ability to embed normal data into a latent space, and vice-versa. The model learns to distinguish between normal data and the synthetically generated anomalies.
  3. Weighted Prediction: The final anomaly score is a weighted combination of the ND block's output (probability of being a noisy sample) and the AE block's output (related to reconstruction error). The optimal weight w is determined empirically, with w=0.2 found to be effective.

Datasets

The paper evaluates AnoRand on a wide range of datasets:

  • Synthetic Datasets: Generated using scikit-learn's make_classification module, with an imbalance rate of 5%.
  • Real-World Datasets:
    • ADBench Benchmark: A collection of 57 datasets covering various domains including NLP, healthcare, science, and general documents.
  • Image Classification Datasets: Including MNIST, CIFAR10, and MVTec.
  • NLP Datasets: Including speech and Imdb.
  • Healthcare Datasets.
    • Other Datasets.

The paper also compares AnoRand to 17 state-of-the-art unsupervised anomaly detection algorithms and 16 supervised anomaly detection algorithms.

Key Insights and Ideas

  • Leveraging Imbalance: The paper effectively exploits the inherent challenge of imbalanced data in anomaly detection by using a Feed-Forward Perceptron, which is known to perform well in such scenarios, to guide the autoencoder.
  • Synthetic Data as a Bridge: Creating synthetic anomalies is a clever way to generate labeled data for semi-supervised training when real labels are scarce. The method's robustness to the noise level used for generation is a significant finding.
  • Synergy of AE and FFP: The joint optimization of the autoencoder and FFP blocks creates a synergistic effect, where each component enhances the other's ability to learn the normal data distribution and identify deviations.
  • AUC PR as a Better Metric: The authors highlight the importance of using AUC PR (Area Under Precision-Recall Curve) over AUC ROC for imbalanced datasets, as AUC ROC can be misleading.

Key Results

  • State-of-the-Art Performance: AnoRand consistently outperforms 17 state-of-the-art unsupervised anomaly detection methods on both synthetic and 57 real-world benchmark datasets, achieving the best AUC PR and AUC ROC in the vast majority of cases.
  • Competitive Supervised Performance: When tested in a fully supervised manner (using actual labels), AnoRand also achieves very good performance, comparable to many state-of-the-art supervised algorithms.
  • Robustness to Noise: The model's performance generally increases with the noise level used for synthetic label generation, suggesting a degree of robustness. A standard deviation of Gaussian noise greater than 0.52 is recommended for optimal performance.
  • Optimal Weighting: An optimal weight w=0.2 for the noise detection block was found to balance the contributions of both components effectively.
  • Training Time: While highly effective, AnoRand's training time is noted to be longer than most other algorithms.

Limitations and Future Work

  • Training Time: The paper acknowledges that the training time for AnoRand is longer compared to many other algorithms. This is a potential limitation for applications requiring very fast model training.
  • Hyperparameter Tuning: While the authors state they did not extensively tune hyperparameters to demonstrate the method's robustness, further optimization could potentially yield even better results.
  • Future Work: The paper doesn't explicitly detail future work, but the success of AnoRand suggests potential avenues for exploring different synthetic data generation techniques, more advanced deep learning architectures, and applications in specific domains where anomaly detection is critical.

Summary updated

Abstract

Anomaly detection or more generally outliers detection is one of the most popular and challenging subject in theoretical and applied machine learning. The main challenge is that in general we have access to very few labeled data or no labels at all. In this paper, we present a new semi-supervised anomaly detection method called \textbf{AnoRand} by combining a deep learning architecture with random synthetic label generation. The proposed architecture has two building blocks: (1) a noise detection (ND) block composed of feed forward ferceptron and (2) an autoencoder (AE) block. The main idea of this new architecture is to learn one class (e.g. the majority class in case of anomaly detection) as well as possible by taking advantage of the ability of auto encoders to represent data in a latent space and the ability of Feed Forward Perceptron (FFP) to learn one class when the data is highly imbalanced. First, we create synthetic anomalies by randomly disturbing (add noise) few samples (e.g. 2\%) from the training set. Second, we use the normal and the synthetic samples as input to our model. We compared the performance of the proposed method to 17 state-of-the-art unsupervised anomaly detection method on synthetic datasets and 57 real-world datasets. Our results show that this new method generally outperforms most of the state-of-the-art methods and has the best performance (AUC ROC and AUC PR) on the vast majority of reference datasets. We also tested our method in a supervised way by using the actual labels to train the model. The results show that it has very good performance compared to most of state-of-the-art supervised algorithms.
BibTeX
@misc{mayaki2023anorand,
  author = {Mansour Zoubeirou A. Mayaki and Michel Riveill},
  title = {{AnoRand: A Semi Supervised Deep Learning Anomaly Detection Method by Random Labeling}},
  year = {2023},
  eprint = {2305.18389},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2305.18389},
}
Figure