Embedding Predictive Adaptation and Refinement for Multi-Modal Industrial Anomaly Detection

Jaeho Cho, Jinsik Bang, Taehwan KimarXiv 2026 · arXiv

TL;DR

EPAR (Embedding Predictive Adaptation and Refinement) is a new framework that significantly boosts multi-modal industrial anomaly detection by bridging the gap between general-purpose AI models and specialized industrial data. By combining a spatial-aware feature adaptation strategy with a "Noise Suppression Network" that learns to distinguish relational discrepancies, EPAR achieves state-of-the-art results in both identifying defective parts and precisely localizing the defects.

What Problem Does This Paper Address?

Industrial Anomaly Detection (IAD) is a high-stakes field where the goal is to find rare, unpredictable defects in manufactured parts using RGB images and 3D point clouds. Because defects are so rare, most systems are "unsupervised"—they learn what a "normal" part looks like and flag anything that deviates.

Current state-of-the-art methods typically use "memory banks" to store features of normal parts. However, they face two major hurdles:

  1. The Domain Gap: Most models are pretrained on general datasets like ImageNet. When applied to specific industrial parts, these "frozen" encoders can’t distinguish subtle anomalies well. Existing adaptation techniques often lose the spatial context needed for precise localization.
  2. Coarse Localization: Anomaly maps are usually interpolated from patch-level scores, resulting in "blurry" or coarse heatmaps that lack fine-grained detail and can trigger false positives.

Methods

The authors propose EPAR, which tackles these issues in two distinct stages:

1. Feature Adaptation (The "Predictive" Part)

Instead of using frozen encoders or simple contrastive learning, EPAR adapts encoders by predicting the embeddings of masked regions based on their surrounding context. This approach, inspired by Joint-Embedding Predictive Architectures (JEPA), forces the model to understand the spatial relationships within a part.

To prevent the model from "collapsing" (outputting the same feature for everything), they use Variance-Invariance-Covariance (VIC) regularization. Uniquely, they apply this at the patch level rather than the batch level, which aligns better with how memory-bank methods actually compare query patches to normal prototypes.

Figure 2. Feature Adaptation Stage. (a) Input Preparation. For 3D point clouds, patches are formed via Farthest Point Sampling (FPS) [19] (1024 centers) and KNN (128 points per patch). Target blocks (distinct colors) are sampled from foreground, with the remaining regions forming the context input. (b) Adaptation Process. The context encoder extracts features Ck from visible regions at layers k ∈{low, mid, high}, while the target encoder processes the complete input to provide target representations. Layer-specific predictors Φk predict masked regions from context features with learnable mask tokens, enabling multi-scale adaptation. (c) VIC Regularization. We integrate VIC regularization [4] at the patch-level with mask prediction: the variance term maintains feature diversity, the invariance term enforces ci-ti consistency, and the covariance term decorrelates feature dimensions.

2. Noise Suppression Network (The "Refinement" Part)

To sharpen the coarse anomaly maps, the authors introduce a Noise Suppression Network (NSN). Unlike previous refinement methods that try to learn what a defect looks like (which fails when real defects look different from synthetic ones), the NSN learns the relational discrepancy between a query patch and its nearest normal neighbor in the memory bank. Because it learns the relationship rather than the appearance, it generalizes remarkably well to unseen, real-world defects.

Datasets

The researchers evaluated EPAR on three major multi-modal benchmarks:

  • MVTec 3D-AD: The standard benchmark for 3D anomaly detection.
  • Eyecandies: A dataset featuring complex geometries and various lighting conditions.
  • PD-REAL: A challenging dataset captured with real-world sensors (RealSense) under varying conditions, used to test generalizability.

Key Insights and Ideas

  • Spatial Context is King for Localization: The paper demonstrates that mask-prediction-based adaptation is superior to contrastive learning for anomaly detection because it preserves the local spatial dependencies required to pinpoint exactly where a defect is.
  • Relationships > Appearance: The NSN's success stems from a "relational signal." By training the network to recognize the difference between a patch and its "normal" counterpart, the system becomes robust to the specific textures or shapes used in synthetic training data.
  • Patch-Level Granularity: A key "aha moment" was realizing that VIC regularization should be applied across patches within a single sample. This ensures that each patch feature encodes distinct, diverse information, which is exactly what a nearest-neighbor search needs to work effectively.
Figure 1. EPAR at a glance. EPAR comprises a feature adap- tation stage and a refinement stage. (top) For each query patch feature, we measure the distance to its nearest normal prototype in the memory bank, shown as densities normalized to unit area for normal (blue curve) and anomalous (red curve) patches. The shaded region marks their overlap, quantified by the overlap co- efficient (lower is better). After adaptation, the overlap narrows for both RGB and 3D, i.e., normal and anomalous patches become better separated. (bottom) Building on the adapted features, our refinement sharpens coarse anomaly maps, suppressing false pos- itives while preserving true anomalies.

Key Results

EPAR sets a new bar for performance, particularly in metrics that matter for real-world deployment:

  • Superior Localization: On MVTec 3D-AD and Eyecandies, EPAR improved AUPRO@1% (a stringent metric that penalizes false alarms) by +1.8% and +1.9%, respectively.
  • Few-Shot Excellence: The model is highly efficient, outperforming all baselines in "few-shot" scenarios where only 5, 10, or 50 normal samples are available for training.
  • Sharper Maps: Qualitative results show that the NSN effectively suppresses "noise" in the background while sharpening the boundaries of actual defects, as seen in Figure 5.
Qualitative comparison showing how EPAR produces much cleaner and more precise anomaly maps than previous methods.

Limitations and Future Work

The authors note a clear accuracy-efficiency trade-off. While their multi-scale feature extraction provides the best results, it increases inference latency by about 1.36x compared to simpler baselines. For ultra-high-speed production lines, a single-scale version of EPAR might be necessary, though it sacrifices some accuracy.

Additionally, while the refinement stage is excellent for pixel-level localization, its impact on sample-level detection (simply saying "this part is broken") is more modest. Future research could explore deeper cross-modal interactions during the adaptation phase to further unify the RGB and 3D representations.

Summary updated

Abstract

Industrial Anomaly Detection (IAD) has evolved from 2D image-based approaches to multi-modal methods that leverage the complementary nature of RGB images and 3D point clouds. Among these, memory bank-based methods, which identify anomalies by the discrepancy between each query patch feature and its nearest normal prototype, have proven particularly effective. However, two limitations remain: (i) employing frozen pretrained encoders yields insufficient discrimination between normal and anomalous features due to the domain gap, while existing contrastive learning-based adaptation struggles to capture the spatial relationships required for precise localization; and (ii) pixel-level anomaly maps interpolated from patch-level distance scores remain coarse, lacking fine-grained detail. To address these limitations, we propose EPAR (Embedding Predictive Adaptation and Refinement). We adapt encoders by predicting the embeddings of masked regions to capture spatial relationships, together with patch-level Variance-Invariance-Covariance (VIC) regularization, resolving the poor discriminability between normal and anomalous features caused by the domain gap. We further propose a Noise Suppression Network that refines coarse maps by learning the relational discrepancy between each patch feature and its nearest normal prototype, rather than the anomaly's appearance. This relational signal remains valid for unseen real defects, despite being learned from synthetic anomalies. EPAR demonstrates strong practical applicability, excelling in few-shot settings and improving AUPRO@1% on MVTec 3D-AD and Eyecandies by +1.8% and +1.9%, respectively.
BibTeX
@misc{cho2026embedding,
  author = {Jaeho Cho and Jinsik Bang and Taehwan Kim},
  title = {{Embedding Predictive Adaptation and Refinement for Multi-Modal Industrial Anomaly Detection}},
  year = {2026},
  eprint = {2611.09521},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.09521},
}
Figure