Template-guided Hierarchical Feature Restoration for Anomaly Detection

Hewei Guo, Liping Ren, Jingjing Fu, Yuwang Wang, Zhizheng Zhang, Cuiling Lan, Haoqian Wang, Xinwen Hou

Abstract

Anomaly detection in complex, multi-object scenes remains a significant challenge for traditional embedding-based and reconstruction-based methods. This paper introduces Template-guided Hierarchical Feature Restoration (THFR), a framework that detects anomalies by restoring image features to a normal state using a retrieved template image. By integrating global and local bottlenecks, THFR effectively identifies both fine-grained structural defects and large-scale logical anomalies. Experimental results on the MVTec LOCO AD and MVTec AD benchmarks demonstrate that THFR achieves state-of-the-art performance, particularly in complex industrial inspection scenarios.

Introduction

Real-world industrial inspection often involves "complicated normal patterns," such as a breakfast box with specific arrangements of fruit or a PCB with numerous components. Existing anomaly detection (AD) algorithms typically struggle with these scenarios. Embedding-based methods, which compare local patches to a memory bank, excel at finding tiny structural defects like scratches but often fail to detect "logical" errors, such as a missing component or an item in the wrong location. Conversely, reconstruction-based methods attempt to rebuild images from compressed representations but frequently suffer from "defect leakage"—where the anomaly itself is reconstructed—or produce blurry results that obscure fine details.

THFR addresses these limitations by shifting the focus from simple reconstruction to feature restoration. By utilizing a reference "template" image, the framework ensures that restored features are strictly anomaly-free, bridging the gap between structural and logical anomaly detection.

Methodology

The THFR framework operates through three primary stages: hierarchical feature extraction, a dual-bottleneck filtering process, and template-guided compensation.

The Template-guided Hierarchical Feature Restoration (THFR) framework, illustrating the hierarchical extraction, G-L bottlenecks, and template-guided compensation modules.

Hierarchical Feature Extraction

The system employs a pre-trained backbone (e.g., ResNet50) to extract multi-level features from the input image. This hierarchical approach allows the model to capture information at various scales, from fine textures to global layouts.

The G-L Bottleneck

To filter out anomalous information, features are processed through two specialized bottlenecks:

  • Global Bottleneck (GBN): Utilizes global max-pooling to compress features into a semantic vector. This forces the network to focus on the overall layout and ignore local misalignments, making it highly effective for detecting logical anomalies.
  • Local Bottleneck (LBN): Employs convolutions to maintain spatial details while filtering out small-scale noise and irregularities, which is crucial for identifying structural anomalies.

Template-guided Compensation

The core innovation of THFR is the use of a Template Bank containing normal images. For a given test image, the system performs an Image-level Nearest Neighbor Search (NNS) to identify the most similar normal template. A "Relation Representation" mechanism then maps the test features toward the template features. This process effectively "restores" distorted or anomalous regions to their expected normal state based on the "gold standard" provided by the template.

Anomalies are detected by calculating the cosine distance between the original pre-trained features and the restored, anomaly-free features.

Results and Discussion

The authors evaluated THFR on the MVTec LOCO AD dataset, designed for logical constraints, and the standard MVTec AD benchmark.

Qualitative comparison of anomaly localization results between PatchCore, RD4AD, and the proposed THFR method across various categories.

On the challenging MVTec LOCO AD dataset, THFR achieved state-of-the-art (SOTA) performance:

  • Localization (sPRO): 74.1, significantly outperforming previous methods like GCAD (70.1) and PatchCore (64.7).
  • Detection (AUROC): 86.0, representing a 2.7% absolute gain over existing benchmarks.

Key insights from the study include the synergy between global and local bottlenecks; large defects require high compression (GBN) to be erased, while small defects need spatial precision (LBN). Furthermore, performing restoration in the high-dimensional feature space avoids the blurriness typical of pixel-level autoencoders, leading to sharper localization.

Conclusion

THFR represents a significant advancement in anomaly detection for complex scenes. By combining hierarchical feature restoration with template guidance and a dual-bottleneck architecture, it successfully addresses both structural and logical anomalies. While the method currently depends on the quality of the retrieved template, future work may explore more sophisticated template synthesis or selection techniques to handle datasets with high intra-class variance.

Summary updated

Abstract

Targeting for detecting anomalies of various sizes for complicated normal patterns, we propose a Template-guided Hierarchical Feature Restoration method, which introduces two key techniques, bottleneck compression and template-guided compensation, for anomaly-free feature restoration. Specially, our framework compresses hierarchical features of an image by bottleneck structure to preserve the most crucial features shared among normal samples. We design template-guided compensation to restore the distorted features towards anomaly-free features. Particularly, we choose the most similar normal sample as the template, and leverage hierarchical features from the template to compensate the distorted features. The bottleneck could partially filter out anomaly features, while the compensation further converts the reminding anomaly features towards normal with template guidance. Finally, anomalies are detected in terms of the cosine distance between the pre-trained features of an inference image and the corresponding restored anomaly-free features. Experimental results demonstrate the effectiveness of our approach, which achieves the state-of-the-art performance on the MVTec LOCO AD dataset.
BibTeX
@inproceedings{guo2023template,
  author = {Hewei Guo and Liping Ren and Jingjing Fu and Yuwang Wang and Zhizheng Zhang and Cuiling Lan and Haoqian Wang and Xinwen Hou},
  title = {{Template-guided Hierarchical Feature Restoration for Anomaly Detection}},
  booktitle = {IEEE/CVF International Conference on Computer Vision},
  year = {2023},
  doi = {10.1109/iccv51070.2023.00593},
}
Figure