Towards a Hierarchical Bayesian Model of Multi-View Anomaly Detection

Zhen Wang, Chao LanIJCAI 2020 · DOI

TL;DR

This paper introduces a novel inductive, semi-supervised Bayesian model for detecting anomalies that manifest inconsistently across multiple views of data. By leveraging a hierarchical Bayesian approach and variational inference, the model effectively learns normal data patterns and identifies outliers, outperforming existing methods on various benchmarks.

What Problem Does This Paper Address?

Traditional anomaly detection methods often struggle with "multi-view anomalies," where an instance might appear normal in one data representation (view) but anomalous in another. For example, an image might be classified as a "car" in one view but be associated with a website about "animals" in another, indicating an inconsistency. Existing multi-view anomaly detection techniques are often transductive and unsupervised, meaning they require all data upfront and don't leverage labeled normal data. This limits their ability to learn robust models of normality and can lead to inefficiencies when new data arrives. This paper aims to address these limitations by proposing a semi-supervised, inductive Bayesian approach.

Methods

The core of this paper is a novel hierarchical Bayesian model for multi-view anomaly detection. The approach begins by defining a generative model for normal data, assuming that different views of an instance are generated from a single underlying latent factor. This is followed by building a hierarchical Bayesian model where priors are placed on model parameters and latent variables, and then priors are placed on those priors. To handle the complexity, variational inference is employed to approximate the posterior distribution of the model's unobserved variables and parameters.

Key methodological contributions include:

  • Inductive Semi-Supervised Learning: Unlike transductive methods, this model can learn from labeled normal data and then detect anomalies in new, unseen data without retraining.
  • Hierarchical Bayesian Framework: This allows for a more robust modeling of uncertainty and parameter estimation by introducing prior distributions over priors.
  • Automatic Relevance Determination (ARD) Prior: Used on projection matrices to automatically determine the dimensionality of the latent space, promoting sparsity and efficiency.
  • Student's-t Distributions: Employed for latent factor priors and likelihoods to enhance the robustness of the estimator against outliers.
  • Variational Inference: Used to derive an analytical approximation to the posterior distribution, making the model computationally tractable.
  • Outlier Score Calculation: Anomaly scores are derived from the negative unscaled Student's-t density of the observed multi-view data, reflecting how likely the data is to be generated from the learned normal model.

Datasets

The paper evaluates the proposed model on:

  • Synthetic Multi-View Settings: Nine datasets from the ODDS library (thyroid, annthyroid, forest cover, vowels, pima, vertebral, lympho, wine, and glass) were used. Views were generated by randomly splitting features.
  • Real-World Multi-View Data:
    • WebKB dataset (Cornell subset): Used for evaluating performance on a real-world web page dataset with four views (content, inbound link, outbound link, cites).
  • MovieLens small dataset: Used for qualitative analysis of movie recommendation data, examining inconsistencies between movie genre and user ratings.

Key Insights and Ideas

  • Leveraging Labeled Normal Data is Crucial: The semi-supervised nature of the model, which utilizes labeled normal data, significantly improves its ability to capture the true nature of normality, leading to better anomaly detection compared to unsupervised methods.
  • Hierarchical Bayesian Modeling Enhances Robustness: The hierarchical structure, combined with robust priors like Student's-t distributions, makes the model less sensitive to noise and better at handling complex data distributions.
  • Inductive Learning is Key for Scalability: The inductive nature of the model allows it to generalize to new data without the need for costly retraining, a significant advantage over transductive approaches.
  • Theoretical Guarantees Matter: The paper provides theoretical analysis, including consistency of the Bayesian estimator and sample complexity bounds, which lend strong credibility to the proposed method.

Key Results

  • Consistent Outperformance: The proposed Bayesian detector consistently outperformed state-of-the-art multi-view anomaly detectors (HOAD, AP, PLVM, LDSR) across various synthetic and real-world datasets for all three types of multi-view anomalies (attribute, class, and class-attribute outliers).
  • High AUC Scores: On the WebKB dataset, the model achieved significantly higher AUC scores compared to competitors, demonstrating its effectiveness on real-world data.
  • Robustness to Outlier Ratios: The model showed robust performance even as the percentage of outliers in the data increased.
  • Qualitative Analysis: The MovieLens experiment provided a compelling qualitative example, identifying movies with high anomalous scores due to inconsistencies between genre and user rating patterns.
  • Theoretical Guarantees: The paper provides theoretical backing for the model's consistency and sample complexity, offering a PAC-style guarantee for multi-view anomaly detection.

Limitations and Future Work

  • Computational Cost: While variational inference is used, the computational complexity for very high-dimensional data or a large number of views might still be a concern. The paper notes that the PLVM method failed to execute on high-dimensional data due to exponent overflow.
  • Hyperparameter Sensitivity: Although ARD priors help with dimensionality, the choice of hyperparameters for the priors might still influence performance, though the paper suggests broad initializations.
  • Future Work: The paper suggests exploring more sophisticated generative models for normal data and investigating the application of the proposed framework to other domains requiring multi-view anomaly detection. Further research into optimizing the variational inference process for even greater efficiency is also a potential avenue.

Summary updated

BibTeX
@inproceedings{wang2020towards,
  author = {Zhen Wang and Chao Lan},
  title = {{Towards a Hierarchical Bayesian Model of Multi-View Anomaly Detection}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2020},
  doi = {10.24963/ijcai.2020/335},
}
Figure