ALGAN: Anomaly Detection by Generating Pseudo Anomalous Data via Latent Variables

Hironori Murase, Kenji Fukumizu

TL;DR

ALGAN (Anomalous Latent variable GAN) improves anomaly detection by training a discriminator to distinguish real normal data from "pseudo-anomalous" data generated by the model itself. By using high-variance latent variables and a buffer of immature samples, ALGAN achieves state-of-the-art accuracy while delivering a massive 10x to 50x speedup in prediction time compared to other image-based methods.

What Problem Does This Paper Address?

Anomaly detection is a "needle in a haystack" problem: anomalous data (like a crack in a tile or a tumor in an X-ray) is rare and difficult to collect. Consequently, most models are trained using only "normal" data.

The authors identify a specific weakness in using standard Generative Adversarial Networks (GANs) for this task. In a standard GAN, the discriminator is trained to distinguish between "real-normal" and "fake-normal" data. As training progresses, these two classes become nearly identical, meaning the discriminator doesn't actually learn a useful boundary for identifying anomalies—it just learns to be a very picky critic of the generator's realism. ALGAN aims to fix this by explicitly providing the discriminator with examples of what "bad" data might look like, without actually needing real anomalous samples.

Methods

The core innovation is the Anomalous Latent variable Generative Adversarial Network (ALGAN). It introduces two types of "pseudo-anomalous" data during training:

  1. Fake-Anomalous Data ($x'_a$): While normal GANs sample latent variables from a standard distribution $N(0, I)$, ALGAN also samples from an "anomalous" latent distribution with much higher variance ($N(0, \sigma^2 I)$, where $\sigma=4$). This forces the generator to produce data that is "off-model"—slightly degraded or distorted versions of the normal data.
  2. Buffered Data ($\hat{x}$): The authors store samples generated during the early, "immature" stages of training. These samples are naturally noisy and unrealistic, serving as another proxy for anomalies.

The discriminator is then trained to classify real-normal data as "Normal" (probability 1) and everything else—fake-normal, fake-anomalous, and buffered data—as "Anomalous" (probability 0). This forces the discriminator to learn a tight decision boundary around the cluster of real normal data.

Datasets

The authors evaluated ALGAN on three primary benchmarks:

  • MVTec-AD: The industry standard for visual inspection, containing 5,354 images across 15 categories (e.g., "bottle," "leather," "tile") with various defects.
  • Magnetic Tile Defects (MTD): A dataset of grayscale images focusing on industrial surface defects like cracks and blowholes.
  • COIL-100: A dataset of 100 different objects. The authors used 10 categories as "normal" and the remaining 90 as "anomalous" to test the model's ability to distinguish between entirely different object classes.

Key Insights and Ideas

  • The Latent Space as a Proxy: The most elegant "aha" moment is the realization that the generator's latent space can be exploited to create anomalies. By pushing the latent variables into high-entropy regions (the "tails" of the distribution), the generator naturally produces outputs that look almost like the target but are fundamentally "wrong."
  • Discriminator-Centric Detection: Unlike many GAN-based methods that rely on "reconstruction error" (how well a model can recreate an image), ALGAN uses the discriminator's raw output as the anomaly score. This is much faster because it avoids the iterative optimization or complex encoding-decoding steps required to calculate reconstruction errors.
  • Stability through Diversity: By mixing "fake-normal" and "fake-anomalous" data, the model avoids the instability often seen in GAN-based anomaly detection, where the model might "forget" what normal looks like as it tries to define the anomaly boundary.

Key Results

  • Inference Speed: This is the headline result. ALGAN-image achieved a prediction time of 10.6 ms per image, making it 10.4 to 54.6 times faster than other image-based SOTA methods like GANomaly or ALOCC.
  • Accuracy: On the MVTec-AD dataset, ALGAN achieved a mean AUROC (Area Under the Receiver Operating Characteristic) of 0.878, significantly outperforming other image-based GAN methods (which averaged around 0.75–0.79).
  • Versatility: The paper demonstrates that the ALGAN architecture works effectively whether you are training directly on raw pixels (ALGAN-image) or on high-level features extracted from a pre-trained model like ResNet (ALGAN-feature).

Limitations and Future Work

  • Geometric Sensitivity: The model struggled with categories like "Hazelnut" and "Screw" where the orientation of the object varies. Because the generator learns a specific distribution, it might flag a rotated (but otherwise perfect) object as an anomaly.
  • Hyperparameter Tuning: The performance is sensitive to the choice of $\sigma$ (the variance of the anomalous latent variables) and the ratios of fake-to-buffered data. Finding the "sweet spot" for a new dataset requires careful validation.
  • Beyond Images: The authors suggest that while this study focused on images, the ALGAN framework could theoretically be applied to other data types, such as time-series signals or text, which remains an open area for future research.

Summary updated

Abstract

In many anomaly detection tasks, where anomalous data rarely appear and are difficult to collect, training using only normal data is important. Although it is possible to manually create anomalous data using prior knowledge, they may be subject to user bias. In this paper, we propose an Anomalous Latent variable Generative Adversarial Network (ALGAN) in which the GAN generator produces pseudo-anomalous data as well as fake-normal data, whereas the discriminator is trained to distinguish between normal and pseudo-anomalous data. This differs from the standard GAN discriminator, which specializes in classifying two similar classes. The training dataset contains only normal data; the latent variables are introduced in anomalous states and are input into the generator to produce diverse pseudo-anomalous data. We compared the performance of ALGAN with other existing methods on the MVTec-AD, Magnetic Tile Defects, and COIL-100 datasets. The experimental results showed that ALGAN exhibited an AUROC comparable to those of state-of-the-art methods while achieving a much faster prediction time.
BibTeX
@article{murase2022algan,
  author = {Hironori Murase and Kenji Fukumizu},
  title = {{ALGAN: Anomaly Detection by Generating Pseudo Anomalous Data via Latent Variables}},
  journal = {IEEE Access},
  year = {2022},
  doi = {10.1109/access.2022.3169594},
}

Loading related content...

Figure