Self-Supervised Anomaly Segmentation via Diffusion Models with Dynamic Transformer UNet

Komal Kumar, Snehashis Chakraborty, Dwarikanath Mahapatra, Behzad Bozorgtabar, Sudipta RoyWACV 2025 · DOI

Abstract

Anomaly segmentation is a critical task in medical and industrial imaging, yet it is often hampered by the scarcity of annotated data. This paper introduces a novel self-supervised framework that leverages Denoising Diffusion Probabilistic Models (DDPMs) enhanced by two key innovations: a multi-frequency "Tsimplex" noise function and a Dynamic Transformer UNet (DTUNet). By utilizing structured noise instead of standard Gaussian noise, the model more effectively repairs large-scale structural anomalies. Our approach achieves a significant performance improvement, outperforming existing generative methods by over 10% in Dice coefficient across multiple benchmarks.

Introduction: The Challenge of Anomaly Detection

In domains like medical imaging, identifying tumors or lesions is vital. However, training supervised models requires vast amounts of pixel-perfect annotations, which are costly and difficult to obtain. Self-supervised anomaly detection (SSAD) offers a solution by training exclusively on "normal" data. The model learns to "repair" noised images back to a healthy state; when presented with an anomaly, the model replaces it with healthy tissue, and the difference between the original and repaired image reveals the defect.

Standard diffusion models typically rely on Gaussian "white noise," which is effective for pixel-level jitter but struggles with large structural anomalies. Furthermore, standard Vision Transformers (ViTs) often lack the spatial inductive biases necessary for precise segmentation.

Methodology

Tsimplex Noise for Structural Repair

The authors propose Tsimplex noise, a four-dimensional Simplex noise variant (spatial, temporal, and batch dimensions). Unlike Gaussian noise, which contains all frequencies equally, Simplex noise is "colored" or structured, possessing more low-frequency components that resemble natural textures.

Comparison of Gaussian and Tsimplex noise across different scales.

As shown in the figure above, Tsimplex noise creates more coherent, blob-like structures compared to the fine-grained nature of Gaussian noise. This forces the diffusion model to learn global anatomical structures to successfully "repair" the corrupted regions, making it far more effective at identifying large-scale anomalies.

Dynamic Transformer UNet (DTUNet)

To address the limitations of standard Transformers, the authors developed the DTUNet. This architecture incorporates:

  • Dynamic Multi-Head Attention (DMHA): Uses a "Head token" to facilitate interaction between attention heads, embedding the spatial inductive biases typically found in CNNs.
  • Hybrid Feedforward (HFF): Employs channel attention to consolidate features, ensuring that fine-grained pixel details are preserved during the denoising process.

Partial Diffusion Process

To improve efficiency, the framework utilizes a partial Markov process. Instead of diffusing to pure noise over 1,000 steps, the model operates within an optimal "sub-range" of steps. This retains enough global context to repair anomalies without the computational burden of full image reconstruction.

Key Results

The proposed method was evaluated on the BrainMRI (NFBS), AnnoBrats (BraTS 2021), and MVTec AD datasets. It consistently outperformed state-of-the-art generative models like AnnoDDPM.

Qualitative comparison of anomaly segmentation results on brain MRI scans.

Quantitatively, the model achieved an average improvement of +10.1% in Dice coefficient, +10.4% in IOU, and +9.6% in AUC. On the industrial MVTec dataset, it reached an average AUROC of 99.17%, rivaling specialized inspection models. Qualitatively, as seen in the figure above, the Tsimplex-based approach produces much cleaner and more accurate segmentation masks compared to standard simplex methods, effectively capturing both large tumors and subtle, high-intensity anomalies.

Conclusion

By integrating structured Tsimplex noise with a spatially-aware Transformer architecture, this research significantly advances the state of self-supervised anomaly segmentation. The ability to repair structural defects by leveraging low-frequency noise components proves to be a powerful paradigm shift from traditional Gaussian-based diffusion. Future work will focus on optimizing sampling speeds and refining hyperparameter selection for diverse imaging modalities.

Summary updated

BibTeX
@inproceedings{kumar2025self,
  author = {Komal Kumar and Snehashis Chakraborty and Dwarikanath Mahapatra and Behzad Bozorgtabar and Sudipta Roy},
  title = {{Self-Supervised Anomaly Segmentation via Diffusion Models with Dynamic Transformer UNet}},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
  year = {2025},
  doi = {10.1109/wacv61041.2025.00770},
}
Figure