A feature shuffling and restoration strategy for universal unsupervised anomaly detection

Wei Luo, Haiming Yao, Zhenfeng Qiang, Xiaotian Zhang, Weihang Zhang

TL;DR

This paper introduces a novel "Feature Shuffling and Restoration" (FSR) strategy for unsupervised anomaly detection. By randomly shuffling feature blocks and training a network to restore them, FSR forces models to learn global context, effectively overcoming the "identical shortcut" problem that plagues existing methods, especially in complex, real-world industrial settings. This leads to a more robust and universally applicable anomaly detection system.

What Problem Does This Paper Address?

In industrial settings, detecting anomalies (defects, errors) is crucial for quality control. Unsupervised anomaly detection methods aim to learn what "normal" looks like from unlabeled data and flag deviations. A common approach is reconstruction-based: train a model to reconstruct normal data, and it will struggle with anomalies, producing higher reconstruction errors. However, a significant challenge, the "identical shortcut" problem, arises where models can perfectly reconstruct even anomalous regions by simply memorizing input patterns rather than truly understanding the data's semantic content. This issue is exacerbated by the dynamic nature of industrial data, where normal patterns can change over time, leading to models that perform well in one scenario but fail drastically in another (e.g., few-shot, separate, or unified settings). The paper addresses the need for a universal anomaly detection system that performs well across these diverse and evolving industrial scenarios.

Methods

The core innovation is the Feature Shuffling and Restoration (FSR) strategy. Instead of directly reconstructing raw image pixels, FSR uses pre-trained multi-scale features as targets. These features are then partitioned into non-overlapping blocks. A key step involves randomly shuffling a subset of these feature blocks, controlled by a "shuffling rate" (τ). A restoration network, specifically a Vision Transformer (ViT) due to its global attention capabilities, is then trained to restore these shuffled blocks to their original positions. This forces the network to learn long-range dependencies and global contextual information, preventing it from simply memorizing and thus overcoming the identical shortcut. The shuffling rate acts as a hyperparameter to dynamically adjust the difficulty of this restoration task, making it suitable for different data complexities.

Datasets

The paper evaluates its method on two widely used datasets:

  • MVTec AD [1]: A benchmark dataset for unsupervised anomaly detection, featuring 15 categories of industrial objects and textures.
  • BTAD [2]: A dataset comprising three industrial products, known for its intricate texture complexity.

Key Insights and Ideas

  • The "Identical Shortcut" is a Task Problem, Not Just a Model Problem: The authors argue that the core issue isn't solely the model's architecture but the nature of the reconstruction task itself, where input and target are too similar.
  • Global Context is Key: FSR forces the model to learn global semantic information by requiring it to reconstruct shuffled feature blocks, which is crucial for distinguishing subtle anomalies.
  • Shuffling Rate as a Difficulty Tuner: The shuffling rate (τ) is a simple yet effective mechanism to control the difficulty of the FSR task, allowing the model to adapt to varying complexities of normal data distributions across different settings (few-shot, separate, unified).
  • Vision Transformers are Well-Suited for FSR: The paper highlights ViT's multi-head attention mechanism as being superior to CNNs for FSR due to its ability to model inter-block relationships and global context.

Key Results

  • State-of-the-Art Performance: FSR consistently achieves superior anomaly detection and localization results across few-shot, separate, and unified settings on both MVTec AD and BTAD datasets.
  • Robustness Across Settings: Unlike previous methods that excel in one setting but degrade in others, FSR demonstrates remarkable robustness, with only marginal performance drops when transitioning between settings. For example, it outperforms UniAD by 1.8/1.2% in the unified setting and exceeds RegAD by 2.3/1.1% in the few-shot setting.
  • Efficiency: FSR achieves a favorable balance between detection accuracy and speed, with a competitive inference time (24.44ms) and lower parameter count and FLOPs compared to some high-performing methods.
  • Qualitative Improvements: Visualizations show that FSR accurately localizes defects and suppresses background noise, unlike methods that produce coarse or noisy anomaly maps. It also effectively reconstructs anomalies as normal patterns, aiding precise segmentation.

Limitations and Future Work

  • Manual Hyperparameter Tuning: The authors acknowledge that optimal performance requires manual tuning of the shuffling rate (τ).
  • Future Work: The paper suggests exploring adaptive adjustments of the shuffling rate based on different settings to automate this tuning process and enhance model adaptability. Additionally, future research will focus on improving the discriminative capability of the pre-trained feature extractor for industrial images to address failure cases caused by subtle anomalies or background noise.

Summary updated

Abstract

Unsupervised anomaly detection is vital in industrial fields, with reconstruction-based methods favored for their simplicity and effectiveness. However, reconstruction methods often encounter an identical shortcut issue, where both normal and anomalous regions can be well reconstructed and fail to identify outliers. The severity of this problem increases with the complexity of the normal data distribution. Consequently, existing methods may exhibit excellent detection performance in a specific scenario, but their performance sharply declines when transferred to another scenario. This paper focuses on establishing a universal model applicable to anomaly detection tasks across different settings, termed as universal anomaly detection. In this work, we introduce a novel, straightforward yet efficient framework for universal anomaly detection: \uline{F}eature \uline{S}huffling and \uline{R}estoration (FSR), which can alleviate the identical shortcut issue across different settings. First and foremost, FSR employs multi-scale features with rich semantic information as reconstruction targets, rather than raw image pixels. Subsequently, these multi-scale features are partitioned into non-overlapping feature blocks, which are randomly shuffled and then restored to their original state using a restoration network. This simple paradigm encourages the model to focus more on global contextual information. Additionally, we introduce a novel concept, the shuffling rate, to regulate the complexity of the FSR task, thereby alleviating the identical shortcut across different settings. Furthermore, we provide theoretical explanations for the effectiveness of FSR framework from two perspectives: network structure and mutual information. Extensive experimental results validate the superiority and efficiency of the FSR framework across different settings.Code is available at https://github.com/luow23/FSR.
BibTeX
@inproceedings{luo2025feature,
  author = {Wei Luo and Haiming Yao and Zhenfeng Qiang and Xiaotian Zhang and Weihang Zhang},
  title = {{A feature shuffling and restoration strategy for universal unsupervised anomaly detection}},
  booktitle = {Knowledge-Based},
  year = {2025},
  doi = {10.1016/j.knosys.2025.114874},
}
Figure