Adapted-MoE: Mixture of Experts with Test-Time Adaption for Anomaly Detection

Tianwu Lei, Silin Chen, Bohan Wang, Zhengkai Jiang, Ningmu Zou

Abstract

Traditional unsupervised anomaly detection (AD) methods often struggle with categories exhibiting high intra-class variation and distribution shifts between training and testing environments. We present Adapted-MoE, a novel framework that employs a Mixture of Experts (MoE) architecture combined with a Test-Time Adaptation (TTA) mechanism. By partitioning a single category into distinct subclasses and aligning unseen test samples with learned distributions, Adapted-MoE effectively eliminates distribution bias and establishes precise decision boundaries. Experimental results on the challenging Texture AD benchmark demonstrate significant improvements in both image-level and pixel-level anomaly detection accuracy.

The Challenge: The "One-Size-Fits-All" Limitation

Most unsupervised AD models operate under the assumption that "normal" data follows a single, cohesive distribution. In practice, however, a single category—such as "Cloth" or "Leather"—often contains numerous variations in texture, color, and pattern.

Figure 1: The problem of using a single decision boundary for diverse distributions.

As illustrated in Figure 1, attempting to draw a single decision boundary around these diverse distributions leads to two primary failures. First, a boundary loose enough to encompass all variations often fails to detect subtle anomalies. Second, distribution bias—the gap between training samples and unseen test samples caused by environmental factors like lighting—can push normal test samples outside the learned boundary, resulting in false positives.

Methodology: Divide, Conquer, and Adapt

Adapted-MoE addresses these issues through a three-stage pipeline designed to handle complex intra-class variations and distribution shifts.

1. Routing Network and Subclass Clustering

Instead of treating the training set as a monolithic block, the framework utilizes a routing network. By applying a Center Loss function, the model maps high-dimensional features from a frozen backbone (e.g., WideResNet50) into distinct regions of a feature space, effectively clustering similar normal samples into subclasses.

2. Mixture of Experts (MoE)

Rather than relying on one large model, Adapted-MoE employs a series of specialized "expert" models, typically implemented as lightweight Multi-Layer Perceptrons (MLPs). Each expert is dedicated to learning the specific decision boundary for one of the identified subclasses. This "divide and conquer" strategy allows for much tighter and more accurate boundaries for each specific texture or pattern.

3. Test-Time Adaptation (TTA)

To bridge the gap between training and testing distributions, the model introduces a TTA mechanism. When an unseen test sample is processed, the routing network identifies the most relevant subclass. The TTA then performs a rapid calibration, adjusting the mean and variance of the test sample's features to align with the distribution the corresponding expert was trained on.

Figure 3: The Adapted-MoE training and inference pipelines.

As shown in Figure 3, this alignment ensures that even if a test sample exhibits slight variations from the training data, it is correctly projected into the expert's discriminating space, neutralizing distribution bias without requiring model retraining.

Experimental Results

The effectiveness of Adapted-MoE was evaluated on the Texture AD benchmark, which includes complex categories like Cloth, Wafer, and Metal. This dataset is specifically designed to test generalization to unseen subclasses.

Key performance metrics include:

  • Image-level AUROC (I-AUROC): The model achieved a significant performance boost, ranging from 2.18% to 7.20% over state-of-the-art baselines like SimpleNet and EfficientAD.
  • Pixel-level AUROC (P-AUROC): Localization accuracy improved by 1.57% to 16.30%, demonstrating superior precision in identifying the exact location of defects.
  • Cloth Dataset Performance: On the particularly diverse Cloth category, Adapted-MoE reached an average I-AUROC of 67.53%, surpassing the previous state-of-the-art of 64.41%.

Conclusion and Future Work

Adapted-MoE demonstrates that a modular, expert-based approach is essential for handling the complexities of real-world anomaly detection. By combining subclass routing with test-time distribution alignment, the framework provides a robust solution to intra-class variation and distribution bias. Future research will focus on optimizing the number of experts to prevent over-division in low-diversity datasets and exploring more sophisticated expert architectures to further enhance detection sensitivity.

Summary updated

BibTeX
@inproceedings{lei2025adapted,
  author = {Tianwu Lei and Silin Chen and Bohan Wang and Zhengkai Jiang and Ningmu Zou},
  title = {{Adapted-MoE: Mixture of Experts with Test-Time Adaption for Anomaly Detection}},
  booktitle = {ICIC},
  year = {2025},
  doi = {10.1007/978-981-96-9891-2\_36},
}
Figure