Multi-Channel Multi-Scale Convolution Attention Variational Autoencoder (MCA-VAE): An Interpretable Anomaly Detection Algorithm Based on Variational Autoencoder

Jingwen Liu, Yuchen Huang, Dizhi Wu, Yuchen Yang, Yanru Chen, Liangyin Chen, Yuanyuan ZhangSensors 2024 · OpenReview

TL;DR

This paper introduces MCA-VAE, a novel anomaly detection system for industrial settings that excels at simultaneously capturing both the temporal patterns within individual sensor data streams and the complex relationships between different sensors. It achieves state-of-the-art performance and, crucially, provides interpretable insights into which sensors are contributing to an anomaly, aiding faster troubleshooting.

What Problem Does This Paper Address?

Modern industrial systems, especially those leveraging IoT, generate vast amounts of multi-dimensional time-series data from numerous sensors. Detecting anomalies in this data is critical for preventing costly downtime and ensuring operational safety. However, existing anomaly detection methods often struggle with two key challenges:

  1. Capturing both temporal and relational features: Many models focus on either the time-dependent patterns within a single sensor's readings (intra-dimensional) or the correlations between different sensors (inter-dimensional), but not both effectively. This can lead to missed anomalies, especially in complex systems where both types of features are important.
  2. Lack of interpretability: Even when anomalies are detected, current methods often fail to pinpoint why an anomaly occurred or which specific sensor(s) are the primary culprits. This "black box" nature makes it difficult for engineers to trust the system and efficiently diagnose and fix problems.

Methods

The proposed MCA-VAE (Multi-Channel Multi-Scale Convolution Attention Variational Autoencoder) tackles these issues with a multi-component architecture:

  • Multi-Scale Weight-Sharing Convolutional Network: This component is designed to extract temporal features from each sensor's data stream. It uses convolutional kernels of different sizes (3, 7, and 15 time units) to capture short, medium, and long-term patterns. The "weight-sharing" aspect is crucial for reducing the number of model parameters, making training more efficient and preventing overfitting, especially in high-dimensional data.
  • Multi-Head Attention Mechanism: This module focuses on capturing the relationships between different sensor dimensions. By allowing the model to "attend" to different sensors simultaneously, it can effectively model their interdependencies.
  • Improved Variational Autoencoder (VAE): The VAE serves as the core anomaly detection engine. It maps the extracted features into a latent space. The authors propose an "improved" VAE that modifies the Evidence Lower Bound (ELBO) to give more weight to reconstruction error, enhancing its ability to learn normal patterns. They also employ a reparameterization trick for faster convergence and numerical stability. Anomalies are detected based on the reconstruction error of the VAE.
  • Anomaly Interpretation: This is a key differentiator. The MCA-VAE leverages the probabilistic nature of the VAE. By analyzing the reconstruction probability of each dimension, it calculates an "anomaly score" for each sensor. These scores are then ranked, providing a clear indication of which sensors are most likely contributing to the detected anomaly.

Datasets

The primary dataset used for training and evaluation is SWaT (Secure Water Treatment) [22]. This dataset is a scaled-down, real-world industrial water treatment plant simulation, featuring 51 sensors and actuators. It includes both normal operation data and data collected during various simulated network attacks. The authors used a sliding window of 30 time points, with each point containing 51 feature values, creating input samples of size 30x51.

Key Insights and Ideas

  • The synergy of temporal and relational features is paramount: The paper strongly suggests that effective anomaly detection in industrial systems requires models that can simultaneously understand how a single sensor behaves over time and how it relates to other sensors.
  • Interpretability is not a luxury, but a necessity: Simply detecting an anomaly is insufficient. Providing engineers with actionable insights into the source of the anomaly is crucial for rapid response and trust in the system.
  • Weight-sharing and multi-scale convolutions are key for efficiency and effectiveness: These techniques allow the model to handle high-dimensional data without becoming computationally prohibitive or prone to overfitting, while still capturing diverse temporal patterns.
  • VAE's probabilistic nature is a goldmine for interpretability: By carefully analyzing the VAE's reconstruction probabilities, the authors unlock a powerful mechanism for localizing anomalies to specific sensors.

Key Results

  • State-of-the-art anomaly detection performance: MCA-VAE achieved an AUC value of 0.982 and an F1 score of 0.905. This represents a 4% improvement in F1 score over the best-performing baseline, TDAD.
  • Accurate anomaly interpretation: The system successfully identified and ranked sensors contributing to anomalies, as demonstrated in Figure 16, providing clear guidance for technicians.
  • Reduced training time: MCA-VAE significantly reduced training time compared to baseline models (e.g., 95 seconds for MCA-VAE vs. 289s for TDAD, 609s for LSTM-NDT, and 1042s for DAGMM), attributed to the efficient weight-sharing mechanism.
  • Balanced performance and resource consumption: Figure 15 shows MCA-VAE achieving high detection accuracy while remaining relatively lightweight in terms of resource consumption compared to other high-performing models.

Limitations and Future Work

  • Limitations: The paper acknowledges that while the model captures temporal and inter-dimensional features, it doesn't explicitly consider spatial dimensional features.
  • Future Work: The authors suggest incorporating spatial dimensional features in the future to provide a more comprehensive understanding of the system's operating status. They also mention the potential for further improvements by introducing more features.

Summary updated

Abstract

With the rapid development of industry, the risks factories face are increasing. Therefore, the anomaly detection algorithms deployed in factories need to have high accuracy, and they need to be able to promptly discover and locate the specific equipment causing the anomaly to restore the regular operation of the abnormal equipment. However, the neural network models currently deployed in factories cannot effectively capture both temporal features within dimensions and relationship features between dimensions; some algorithms that consider both types of features lack interpretability. Therefore, we propose a high-precision, interpretable anomaly detection algorithm based on variational autoencoder (VAE). We use a multi-scale local weight-sharing convolutional neural network structure to fully extract the temporal features within each dimension of the multi-dimensional time series. Then, we model the features from various aspects through multiple attention heads, extracting the relationship features between dimensions. We map the attention output results to the latent space distribution of the VAE and propose an optimization method to improve the reconstruction performance of the VAE, detecting anomalies through reconstruction errors. Regarding anomaly interpretability, we utilize the VAE probability distribution characteristics, decompose the obtained joint probability density into conditional probabilities on each dimension, and calculate the anomaly score, which provides helpful value for technicians. Experimental results show that our algorithm performed best in terms of F1 score and AUC value. The AUC value for anomaly detection is 0.982, and the F1 score is 0.905, which is 4% higher than the best-performing baseline algorithm, Transformer with a Discriminator for Anomaly Detection (TDAD). It also provides accurate anomaly interpretation capability.
BibTeX
@inproceedings{liu2024multi,
  author = {Jingwen Liu and Yuchen Huang and Dizhi Wu and Yuchen Yang and Yanru Chen and Liangyin Chen and Yuanyuan Zhang},
  title = {{Multi-Channel Multi-Scale Convolution Attention Variational Autoencoder (MCA-VAE): An Interpretable Anomaly Detection Algorithm Based on Variational Autoencoder}},
  booktitle = {Sensors},
  year = {2024},
  url = {https://openreview.net/forum?id=HNcK2Yyxkx},
}
Figure