Skip-GANomaly++: Skip Connections and Residual Blocks for Anomaly Detection (Student Abstract)

Juneyoung Park, Jae-Ryung Hong, Min-Hye Kim, Tae-Joon KimAAAI 2024 · DOI

Abstract

Semi-supervised anomaly detection is a critical task in domains where abnormal samples are scarce, such as medical diagnostics and industrial inspection. This paper introduces Skip-GANomaly++, an enhanced Generative Adversarial Network (GAN) framework that integrates residual blocks and Atrous Spatial Pyramidal Pooling (ASPP) to address the limitations of existing models. By stabilizing deep network training and capturing multi-scale contextual information, Skip-GANomaly++ achieves state-of-the-art performance on the CIFAR-10 benchmark, particularly in complex object categories where previous architectures struggled.

Introduction

Anomaly detection aims to identify rare items or events that deviate significantly from the majority of a dataset. In many real-world scenarios, obtaining a balanced dataset of "normal" and "abnormal" samples is nearly impossible. Consequently, researchers rely on semi-supervised learning, where a model is trained exclusively on normal data. The underlying principle is that a model trained to reconstruct normal images will fail to accurately reconstruct an "abnormal" image it has never encountered. The resulting reconstruction error, or "anomaly score," serves as the detection metric.

While previous models like Skip-GANomaly utilized skip connections to preserve fine details, they often faced challenges with training instability as network depth increased and suffered from information loss at the latent bottleneck. Skip-GANomaly++ was developed to create a deeper, more stable architecture that maintains critical spatial information throughout the encoding and decoding process.

Methodology

Skip-GANomaly++ builds upon the GAN framework, consisting of a Generator and a Discriminator. The authors introduced three primary architectural innovations to improve performance and stability:

  1. Residual Units: Inspired by ResNet, residual blocks are implemented throughout the network. These blocks utilize shortcut connections that allow gradients to flow more easily during backpropagation, effectively mitigating the degradation problem common in deep neural networks.
  2. Atrous Spatial Pyramidal Pooling (ASPP): Positioned at the "bottleneck" between the encoder and decoder, the ASPP block uses dilated (atrous) convolutions at multiple rates. This allows the model to capture features at various scales—from fine textures to global structures—preventing the loss of context that typically occurs during image compression.
  3. Skip Connections: Carried over from the original Skip-GANomaly, these connections link early encoder layers to corresponding decoder layers, ensuring that high-resolution spatial details are preserved in the final reconstruction.
The architecture of Skip-GANomaly++ featuring Residual Blocks, ASPP, and Skip Connections.

Experimental Setup

The model was evaluated using the CIFAR-10 dataset. The authors employed a "one-leave-out" protocol where, for each experiment, one class (e.g., "bird") is treated as the anomaly, while the remaining nine classes serve as the normal training data. The training set consisted of 45,000 normal samples, while the test set included 9,000 normal and 6,000 abnormal samples, all resized to 32x32 pixels.

Key Results

Performance was measured using the Area Under the Curve (AUC). Skip-GANomaly++ outperformed established baselines—including EGBAD, AnoGAN, GANomaly, and the original Skip-GANomaly—in 8 out of 10 CIFAR-10 categories.

Comparison of AUC performance across CIFAR-10 classes for various anomaly detection models.

As shown in the results table, the model achieved near-perfect scores in several categories, such as 'ship' (0.998), 'deer' (0.998), and 'frog' (0.999). Most notably, in the 'bird' class, the AUC increased from 0.448 (Skip-GANomaly) to 0.819, demonstrating the model's superior ability to handle complex, organic shapes.

Discussion and Conclusion

The primary insight of this work is that architectural stability enables effective depth. By combining residual units with skip connections, the authors achieved the stability of ResNets alongside the detail-retention of U-Nets. Furthermore, the integration of ASPP at the bottleneck addresses the "bottleneck problem," allowing the model to maintain a global view of the image which is vital for context-dependent anomalies.

While the model slightly underperformed in rigid, highly structured classes like 'car' and 'truck', its overall average performance is significantly more robust. Future work will involve validating this architecture on high-resolution medical datasets, such as MRI or X-ray scans, where multi-scale context is essential for accurate diagnosis.

Summary updated

BibTeX
@inproceedings{park2024skip,
  author = {Juneyoung Park and Jae-Ryung Hong and Min-Hye Kim and Tae-Joon Kim},
  title = {{Skip-GANomaly++: Skip Connections and Residual Blocks for Anomaly Detection (Student Abstract)}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2024},
  doi = {10.1609/aaai.v38i21.30496},
}
Figure