U2AD: Uncertainty-based Unsupervised Anomaly Detection Framework for Detecting T2 Hyperintensity in MRI Spinal Cord

Qi Zhang, Xiuyuan Chen, Ziyi He, Kun Wang, Lianming Wu, Hongxing Shen, Jianqi SunarXiv 2025 · OpenReview

TL;DR

This paper introduces U²AD, a novel framework for unsupervised anomaly detection in spinal cord MRI, specifically for identifying T2 hyperintensities. U²AD addresses the critical challenge of domain shifts in clinical data by training and testing on the same dataset, leveraging uncertainty-guided masking to balance reconstruction accuracy and anomaly detection sensitivity.

What Problem Does This Paper Address?

Diagnosing conditions like degenerative cervical myelopathy often relies on identifying T2 hyperintensities in spinal cord MRI. While deep learning shows promise, supervised methods require extensive annotated data, which is scarce. Unsupervised Anomaly Detection (UAD) offers a solution by learning from normal data, but existing UAD methods struggle with "domain shifts" – differences in data distribution between curated training sets and real-world clinical data. Furthermore, UAD models face a fundamental conflict: they need to accurately reconstruct normal anatomy while simultaneously amplifying errors in anomalous regions for detection. This can lead to either missing anomalies (false negatives) or misidentifying normal variations (false positives).

Methods

U²AD employs a Vision Transformer (ViT)-based architecture within a "mask-and-reconstruction" paradigm. Key innovations include:

  • Uncertainty-Guided Masking: Instead of random masking, U²AD uses uncertainty estimates to guide which image patches are masked during training.
  • Epistemic Uncertainty (EU): Derived from Monte Carlo (MC) sampling, EU quantifies model confidence. High EU regions are prioritized for reconstruction improvement, enhancing robustness in low-confidence areas.
  • Aleatoric Uncertainty (AU): Estimated from reconstruction errors, AU highlights potential anomalies. High AU regions are excluded from training to amplify reconstruction errors in anomalous areas.
  • Two-Stage Adaptation: The framework first pretrains on a large healthy dataset to learn general anatomical representations. Then, it undergoes a two-stage adaptation on clinical data: EU-guided adaptation to improve reconstruction in uncertain regions, followed by AU-guided adaptation to enhance anomaly detection sensitivity.
  • Dual Reconstruction Loss: A combination of Mean Squared Error (MSE) and edge reconstruction loss ensures both pixel accuracy and fine anatomical detail preservation.

Datasets

  • Spine Generic T2w (#1): A large, public dataset of 270 healthy spinal cord MR images used for pretraining.
  • OpenNeuro T2w (#2): A public dataset of 50 3D MR images used to create pseudo-anomalous images for reconstruction assessment.
  • RenjiDCM T2w (#3): A clinical dataset of 125 spinal cord MR images with manual annotations for T2 hyperintensities, used for evaluating reconstruction and anomaly detection.
  • RuijinDCM T2w (#4): A clinical dataset of 95 spinal cord MR images, characterized by a sparsity of anomalies, used for evaluating anomaly detection and localization.

Key Insights and Ideas

  • Uncertainty is Key for Clinical UAD: Directly integrating epistemic and aleatoric uncertainty into the masking and training process is crucial for overcoming domain shifts and the inherent task conflict in UAD.
  • "Mask-and-Reconstruct" with Uncertainty Guidance: The ViT architecture, combined with uncertainty-guided masking, allows the model to learn robust representations of normal anatomy while effectively highlighting deviations.
  • Clinical Data is Different: The performance drop of traditional UAD methods on clinical datasets highlights the need for frameworks that can adapt to real-world data distributions without relying on curated, perfectly clean datasets.
  • Balancing Act: U²AD's success lies in its ability to find an optimal balance between reconstructing normal regions accurately and amplifying reconstruction errors in anomalous regions, a critical challenge in anomaly detection.

Key Results

  • Superior Performance: U²AD outperformed existing UAD methods and supervised approaches on both RenjiDCM T2w and RuijinDCM T2w datasets for patient-level identification and segment-level localization tasks.
  • High Accuracy and Specificity: On RenjiDCM T2w, U²AD achieved the highest accuracy (0.7740), F1 score (0.8336), and specificity (0.5533) for patient-level identification. For segment-level localization, it achieved an F1 score of 0.7345.
  • Robustness on Sparse Data: On the challenging RuijinDCM T2w dataset with sparse anomalies, U²AD demonstrated remarkable robustness, significantly outperforming other models.
  • Effective Reconstruction: U²AD achieved the highest SSIM and RSNR, and lowest MSE and Variance in reconstruction tasks, indicating superior model confidence and accuracy.

Limitations and Future Work

  • Computational Cost: The Monte Carlo estimation of uncertainty increases inference time, though the authors found K=10 to be an acceptable balance.
  • 2D Focus: The current implementation is designed for 2D images. Future work aims to extend U²AD to 3D data for modalities like CT and MRI.
  • Generalizability: The authors plan to validate U²AD's performance on other anomaly detection tasks and datasets in diverse clinical contexts.

Summary updated

Abstract

T2 hyperintensities in spinal cord MR images are crucial biomarkers for conditions such as degenerative cervical myelopathy. However, current clinical diagnoses primarily rely on manual evaluation. Deep learning methods have shown promise in lesion detection, but most supervised approaches are heavily dependent on large, annotated datasets. Unsupervised anomaly detection (UAD) offers a compelling alternative by eliminating the need for abnormal data annotations. However, existing UAD methods rely on curated normal datasets and their performance frequently deteriorates when applied to clinical datasets due to domain shifts. We propose an Uncertainty-based Unsupervised Anomaly Detection framework, termed U2AD, to address these limitations. Unlike traditional methods, U2AD is designed to be trained and tested within the same clinical dataset, following a "mask-and-reconstruction" paradigm built on a Vision Transformer-based architecture. We introduce an uncertainty-guided masking strategy to resolve task conflicts between normal reconstruction and anomaly detection to achieve an optimal balance. Specifically, we employ a Monte-Carlo sampling technique to estimate reconstruction uncertainty mappings during training. By iteratively optimizing reconstruction training under the guidance of both epistemic and aleatoric uncertainty, U2AD reduces overall reconstruction variance while emphasizing regions. Experimental results demonstrate that U2AD outperforms existing supervised and unsupervised methods in patient-level identification and segment-level localization tasks. This framework establishes a new benchmark for incorporating uncertainty guidance into UAD, highlighting its clinical utility in addressing domain shifts and task conflicts in medical image anomaly detection. Our code is available: https://github.com/zhibaishouheilab/U2AD
BibTeX
@misc{zhang2025u,
  author = {Qi Zhang and Xiuyuan Chen and Ziyi He and Kun Wang and Lianming Wu and Hongxing Shen and Jianqi Sun},
  title = {{U2AD: Uncertainty-based Unsupervised Anomaly Detection Framework for Detecting T2 Hyperintensity in MRI Spinal Cord}},
  year = {2025},
  eprint = {2503.13400},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2503.13400},
}
Figure