Group Anomaly Detection Using Deep Generative Models

Raghavendra Chalapathy, Edward Toth, Sanjay Chawla

Abstract

Group anomaly detection (GAD) identifies sets of observations that exhibit irregular collective behavior despite appearing normal individually. This paper leverages Deep Generative Models (DGMs), specifically Variational Autoencoders (VAE) and Adversarial Autoencoders (AAE), to model the "normal" distribution of groups. The proposed approach effectively detects both point-based anomalies (outliers within a group) and distribution-based anomalies (irregular mixtures of normal points). Evaluations on synthetic and image datasets show that DGMs achieve superior performance compared to traditional kernel-based methods, particularly as the number of group observations grows.

Introduction

Traditional anomaly detection research primarily focuses on point anomalies, which are individual data points that deviate significantly from the rest of the dataset. However, many critical real-world applications—ranging from particle physics simulations to the detection of manipulated digital media—require identifying group anomalies. In these cases, individual elements may appear perfectly normal when viewed in isolation, but their collective arrangement or distribution is irregular.

Group anomalies generally fall into two categories:

  1. Point-based anomalies: The group contains individual elements that are themselves outliers (e.g., a few tiger images hidden within a collection of domestic cats).
  2. Distribution-based anomalies: The individual elements are normal, but their mixture or structural arrangement is incorrect (e.g., a collection of cat images where several are rotated, or a single image composed of two different scenes stitched together).

Conventional methods like Mixture Models or One-Class Support Vector Machines (OCSVM) often struggle with the high dimensionality of modern data or require extensive manual feature engineering. This paper proposes a more robust alternative using Deep Generative Models.

Methodology

The authors utilize Deep Generative Models (DGMs) to learn a "group reference" manifold that represents the distribution of normal group behavior. Two primary architectures are explored:

  • Variational Autoencoders (VAE): These models use a probabilistic approach to map input data into a latent space. A Kullback–Leibler (KL) divergence term is used as a regularizer to ensure the latent space follows a predefined distribution.
  • Adversarial Autoencoders (AAE): Similar to VAEs, AAEs map data to a latent space but replace the KL divergence penalty with an adversarial training scheme. A discriminator is trained to distinguish between the model's latent representations and a prior distribution, forcing the encoder to produce more realistic latent codes.

The GAD Workflow

The detection process involves three main steps:

  1. Characterization: An encoder function $f$ maps individual group members into a low-dimensional latent space.
  2. Aggregation: A function $g$ aggregates these latent representations to define a "Group Reference" ($G^{(ref)}$).
  3. Scoring: New groups are evaluated by calculating their distance from the learned reference manifold. Groups that fall far from this manifold are flagged as anomalies.

Experimental Setup

The researchers evaluated the DGM approach against traditional baselines using synthetic data and modified real-world image datasets.

Overview of the input data and the three types of group anomalies (A: point-based, B: mixture-based, C: structural/stitched) evaluated in the study.

As shown in the figure above, the experiments included:

  • CIFAR-10 (Modified): Scenarios such as "Tigers in Cats" (point-based), "Rotated Entities" (distribution-based), and "Cats and Dogs" (mixture-based).
  • Scene Images: Detecting "stitched" images that combine disparate scenes like mountains and cities.
  • Synthetic Rotated Gaussians: A controlled environment to test the models' ability to detect covariance shifts.

Results and Discussion

The experimental results highlight several key advantages of the DGM-based approach:

  • Superior Detection Accuracy: On synthetic datasets, both AAE and VAE achieved perfect AUROC scores (1.0), significantly outperforming OCSVM. In complex image tasks like "Rotated Cats," AAE maintained perfect performance while traditional methods like MGM failed, scoring near chance levels (0.53–0.62).
  • Scaling Behavior: A critical finding was the relationship between data volume and performance. While traditional statistical methods may perform better on very small datasets, DGMs rapidly overtake them as the number of group observations increases, effectively learning the underlying manifold of normal behavior.
  • AAE vs. VAE: The Adversarial Autoencoder generally proved more robust. The authors attribute this to the adversarial training's ability to impose more flexible and complex priors on the latent space compared to the rigid mathematical constraints of the VAE.
  • Inference Efficiency: Although DGMs require more time to train, they are inductive. Once the model is trained, it can process and score new groups much faster than kernel-based methods, which often scale poorly with the number of observations.

Conclusion and Future Work

This research demonstrates that Deep Generative Models provide a powerful framework for Group Anomaly Detection, particularly for high-dimensional image data where distributional shifts are subtle. However, the models remain sensitive to sample size, performing less effectively when the group size is small.

Future research directions include extending these models to handle temporal dynamics using Recurrent Neural Networks (RNNs) to detect anomalies in video or sensor streams, and further automating hyperparameter tuning to reduce the manual effort required for model optimization.

Summary updated

BibTeX
@inproceedings{chalapathy2018group,
  author = {Raghavendra Chalapathy and Edward Toth and Sanjay Chawla},
  title = {{Group Anomaly Detection Using Deep Generative Models}},
  booktitle = {ECML/PKDD},
  year = {2018},
  doi = {10.1007/978-3-030-10925-7\_11},
}
Figure