LEAN-DMKDE: Quantum Latent Density Estimation for Anomaly Detection (Student Abstract)

Joseph A. Gallego-Mejia, Oscar A. Bustos-Brinez, Fabio A. GonzálezAAAI 2023 · DOI

Introduction

Anomaly detection—identifying data points that deviate from "normal" behavior—is a cornerstone of modern machine learning, used in everything from fraud detection to industrial monitoring. Historically, researchers have faced a trade-off: statistical density-estimation methods (like Kernel Density Estimation) offer strong theoretical foundations but struggle with high-dimensional data, while deep learning models (like Autoencoders) excel at learning compact representations but often lack an explicit, robust statistical mechanism for calculating the "likelihood" of a point being normal.

LEAN-DMKDE (Quantum Latent Density Estimation for Anomaly Detection) addresses this gap by proposing an end-to-end architecture that combines the dimensionality reduction of autoencoders with a density estimation model based on density matrices, a formalism borrowed from quantum mechanics. By mapping latent data features into a high-dimensional Hilbert space, the model achieves state-of-the-art performance across a wide variety of benchmark datasets.

Methodology

The LEAN-DMKDE architecture consists of four main stages that integrate deep learning with quantum-inspired statistics:

  1. Autoencoder: The input data is passed through an encoder to create a low-dimensional latent representation ($y$). This step performs dimensionality reduction and captures the most essential features of the data.
  2. Adaptive Fourier Feature (AFF) Layer: This layer maps the latent representation into a high-dimensional Hilbert space. Crucially, the dot product in this space approximates a Gaussian kernel. Unlike static kernel methods, these features are "adaptive," meaning they can be optimized during training.
  3. Quantum Measurement Layer: The model constructs a density matrix ($\rho$), which represents the distribution of the training data in the Hilbert space. To manage memory requirements, the authors use an eigen-decomposition to create a low-rank approximation of the matrix, keeping only the $r$ largest eigenvalues.
  4. Anomaly Detector: For a new data point, the model calculates a density estimate ($\hat{f}(x)$) using the density matrix. If the density is below a specific threshold ($\tau$)—determined by a percentile of the training data—the point is flagged as an anomaly.
The LEAN-DMKDE architecture, illustrating the flow from input data through the autoencoder to the quantum measurement KDE and final anomaly detection.

The entire pipeline is differentiable, allowing it to be trained end-to-end using standard gradient-based optimization.

Experimental Results

The model was evaluated using 18 different anomaly detection datasets sourced from the ODDS Library, covering a variety of domains and data types. The researchers compared LEAN-DMKDE against 11 baseline algorithms, ranging from classical methods like Isolation Forest (iForest) and K-Nearest Neighbors (KNN) to deep learning approaches like DeepSVDD.

Comparison of F1 scores across 18 datasets. LEAN-DMKDE demonstrates superior median performance and higher consistency compared to 11 baseline algorithms.

Experimental results showed that LEAN-DMKDE outperformed the majority of the baseline methods. As shown in the boxplot of F1 scores, LEAN-DMKDE not only achieved a higher median score but also demonstrated more consistent performance across the 18 different datasets compared to its peers.

Key Insights and Discussion

  • Quantum Formalism for Classical Tasks: The use of density matrices provides a sophisticated way to model data distributions. By treating data points as "states" in a Hilbert space, the researchers leverage the mathematical tools of quantum mechanics to perform precise density estimation.
  • Bridging the Gap: The model successfully integrates the feature-learning power of deep learning with the statistical clarity of kernel methods.
  • Efficiency through Approximation: One of the primary hurdles in using density matrices is their memory footprint. The authors' use of matrix eigen-decomposition to create a rank-$r$ approximation makes this approach computationally practical for real-world applications.

Limitations and Future Work

As a condensed research summary, some areas remain for further exploration. The paper does not provide a detailed comparison of training times or inference latency compared to simpler models like Isolation Forest. Additionally, the sensitivity of the model to hyperparameters, such as the latent space dimension and the rank of the density matrix, warrants further analysis. Future work could involve applying this "Quantum Latent" approach to other tasks beyond anomaly detection, such as clustering or generative modeling.

Summary updated

BibTeX
@inproceedings{gallegomejia2023lean,
  author = {Joseph A. Gallego-Mejia and Oscar A. Bustos-Brinez and Fabio A. González},
  title = {{LEAN-DMKDE: Quantum Latent Density Estimation for Anomaly Detection (Student Abstract)}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2023},
  doi = {10.1609/aaai.v37i13.26965},
}
Figure