Unsupervised Energy-based Out-of-distribution Detection using Stiefel-Restricted Kernel Machine

Francesco Tonin, Arun Pandey, Panagiotis Patrinos, Johan A. K. Suykens

Abstract

This paper introduces an unsupervised energy-based framework for Out-of-Distribution (OOD) detection using the Stiefel-Restricted Kernel Machine (St-RKM). By integrating deep autoencoders with manifold-constrained latent space analysis, the proposed method identifies anomalies without requiring labeled data or prior exposure to OOD samples. The approach significantly outperforms existing generative models and supervised energy-based baselines across image and time-series benchmarks.

Introduction

In real-world machine learning deployment, the ability of a model to flag test samples that differ significantly from its training data—Out-of-Distribution (OOD) detection—is a critical safety requirement. Most state-of-the-art OOD detectors rely on supervised classifiers and labeled data, which introduces two major flaws: a dependency on expensive labels and a tendency to be fine-tuned for specific "known" OOD examples, making them less effective at spotting truly novel anomalies.

Researchers at KU Leuven propose a powerful unsupervised alternative: the Stiefel-Restricted Kernel Machine (St-RKM). This model learns the "energy" of a distribution entirely from raw, unlabeled in-distribution data, providing a robust solution for "open-world" settings like autonomous driving or medical diagnostics.

The St-RKM Framework

The St-RKM is a hybrid architecture combining a deep autoencoder with a Kernel Principal Component Analysis (KPCA) objective. A key technical innovation is the use of Manifold Optimization, where the model's interconnection matrix $U$ is constrained to the Stiefel manifold. This ensures that internal feature-matching weights remain orthonormal, preventing model collapse and ensuring a robust representation of the data structure.

Conceptual overview of the mapping from input space to the latent subspace defined by the Stiefel manifold.

The framework defines energy through different components to capture various types of anomalies:

  • $E_{FullEnergy}$: The total objective function, combining reconstruction error and latent space alignment.
  • $E_{kPCAError}$: Focuses specifically on how well a sample fits into the learned latent subspace.
  • $E_{AELoss}$: The standard autoencoder reconstruction error.

Experimental Evaluation

The model was rigorously tested on computer vision benchmarks (Fashion-MNIST as in-distribution vs. MNIST, dSprites, SVHN, and CIFAR-10 as OOD) and time-series data (ECG5000).

One of the most significant findings is the "Closeness Paradox": different energy components excel at different tasks. For OOD data "close" to training data (e.g., MNIST vs. Fashion-MNIST), $E_{kPCAError}$ is the superior detector. For highly distinct OOD data (e.g., CIFAR-10 vs. Fashion-MNIST), $E_{FullEnergy}$ performs better due to the inclusion of reconstruction error.

Density plots showing the separation of in-distribution (blue) and OOD (red) energy scores across various benchmarks and models.

Key Results

  • Superiority over Supervised Baselines: The unsupervised St-RKM outperformed supervised energy-based methods (e.g., Liu et al., 2020) when those methods were not specifically fine-tuned with OOD examples.
  • Low False Positive Rates: On MNIST (OOD), $E_{kPCAError}$ achieved a False Positive Rate at 95% True Positive Rate (FPR95) of 0.38%, compared to 75.97% for the supervised baseline. On CIFAR-10, $E_{FullEnergy}$ achieved an FPR95 of 0.34%.
  • Time Series Performance: The model achieved an AUROC of 94.02% on the ECG5000 dataset, outperforming Variational Autoencoders (VAE) and Generative Adversarial Networks (GAN).

Conclusion

The Stiefel-Restricted Kernel Machine provides a robust, label-agnostic approach to OOD detection. By learning the underlying structure of data rather than just classification boundaries, it offers a more general-purpose solution for anomaly detection. Future work will focus on scaling this approach to high-resolution datasets like ImageNet and investigating its robustness against adversarial attacks.

Summary updated

Abstract

Detecting out-of-distribution (OOD) samples is an essential requirement for the deployment of machine learning systems in the real world. Until now, research on energy-based OOD detectors has focused on the softmax confidence score from a pre-trained neural network classifier with access to class labels. In contrast, we propose an unsupervised energy-based OOD detector leveraging the Stiefel-Restricted Kernel Machine (St-RKM). Training requires minimizing an objective function with an autoencoder loss term and the RKM energy where the interconnection matrix lies on the Stiefel manifold. Further, we outline multiple energy function definitions based on the RKM framework and discuss their utility. In the experiments on standard datasets, the proposed method improves over the existing energy-based OOD detectors and deep generative models. Through several ablation studies, we further illustrate the merit of each proposed energy function on the OOD detection performance.
BibTeX
@inproceedings{tonin2021unsupervised,
  author = {Francesco Tonin and Arun Pandey and Panagiotis Patrinos and Johan A. K. Suykens},
  title = {{Unsupervised Energy-based Out-of-distribution Detection using Stiefel-Restricted Kernel Machine}},
  booktitle = {IJCNN},
  year = {2021},
  doi = {10.1109/ijcnn52387.2021.9533706},
}
Figure