ADFA: Attention-augmented Differentiable top-k Feature Adaptation for Unsupervised Medical Anomaly Detection

Yiming Huang, Guole Liu, Yaoru Luo, Ge Yang

Abstract

The scarcity of annotated data, particularly for rare diseases, limits the variability of training data and the range of detectable lesions, presenting a significant challenge for supervised anomaly detection in medical imaging. To solve this problem, we propose a novel unsupervised method for medical image anomaly detection: Attention-Augmented Differentiable top-k Feature Adaptation (ADFA). The method utilizes Wide-ResNet50-2 (WR50) network pre-trained on ImageNet to extract initial feature representations. To reduce the channel dimensionality while preserving relevant channel information, we employ an attention-augmented patch descriptor on the extracted features. We then apply differentiable top-k feature adaptation to train the patch descriptor, mapping the extracted feature representations to a new vector space, enabling effective detection of anomalies. Experiments show that ADFA outperforms state-of-the-art (SOTA) methods on multiple challenging medical image datasets, confirming its effectiveness in medical anomaly detection.

Introduction

Detecting anomalies in medical images—such as tumors in an MRI or lesions in an ultrasound—is a high-stakes task. While supervised deep learning has made massive strides, it faces significant hurdles when applied to rare diseases. Training a model to find a "needle in a haystack" is difficult when examples of the "needle" are scarce. Furthermore, expert radiologists' time for labeling is expensive and limited.

Unsupervised methods offer a solution by learning only from "normal" images. However, existing approaches like reconstruction-based models or standard embedding-similarity methods often struggle with the high variability and feature redundancy inherent in medical imaging. ADFA addresses these challenges by learning the intricate patterns of healthy tissue and mapping features into a space where anomalies stand out clearly.

Methodology

The ADFA framework consists of three main stages designed to refine raw features into discriminative representations for anomaly detection.

The ADFA framework architecture, illustrating the flow from feature extraction to feature adaptation.

1. Feature Extraction

The process begins with a pre-trained Wide-ResNet50-2 (WR50) backbone. To incorporate spatial context, 2D coordinate channels are concatenated with the extracted feature maps, providing the model with positional information crucial for identifying localized abnormalities.

2. Attention-Augmented Patch Descriptor

Standard pre-trained models often produce redundant features. ADFA employs a custom attention module using both Global Max Pooling (GMP) and Global Average Pooling (GAP) to highlight the most informative channels. This reduces dimensionality while preserving critical diagnostic information, effectively filtering out "noise" that isn't relevant to medical pathology.

3. Differentiable Top-k Adaptation

The core innovation of ADFA is the differentiable top-k operator. Instead of a single global average for "normal," ADFA maintains a center feature bank. During training, a "soft" top-k search identifies the $K$ nearest normal centers. This allows the model to be optimized via gradient descent, pulling normal features closer to their respective neighbors in a new vector space. This multi-center approach prevents the model from being confused by the natural diversity of healthy anatomy (e.g., bone vs. soft tissue).

Experiments and Results

The researchers validated ADFA on four distinct medical imaging benchmarks: BrainMRI (tumors), Covid (chest X-rays), BUSI (breast ultrasound), and SIPaKMeD (cervical pap smears).

Comparison of ADFA against state-of-the-art methods across four medical imaging datasets.

ADFA consistently outperformed or matched state-of-the-art (SOTA) models such as PaDiM, SPADE, and Patch SVDD. It achieved the highest AUROC on the BUSI (0.966) and SIPaKMeD (0.972) datasets. Notably, on the SIPaKMeD dataset, ADFA improved the AUROC by 3.7% over the previous best method. Ablation studies confirmed that the attention module and the differentiable top-k adaptation were both vital to these performance gains.

Conclusion

ADFA provides a robust unsupervised framework for medical anomaly detection by refining how AI "looks" at normal anatomy. By combining channel attention with a differentiable multi-center adaptation strategy, it effectively identifies rare pathologies without requiring extensive labeled datasets. Future work will focus on extending ADFA to anomaly localization, providing clinicians with visual "heatmaps" to pinpoint lesions, and exploring its utility in non-medical domains.

Summary updated

Abstract

The scarcity of annotated data, particularly for rare diseases, limits the variability of training data and the range of detectable lesions, presenting a significant challenge for supervised anomaly detection in medical imaging. To solve this problem, we propose a novel unsupervised method for medical image anomaly detection: Attention-Augmented Differentiable top-k Feature Adaptation (ADFA). The method utilizes Wide-ResNet50-2 (WR50) network pre-trained on ImageNet to extract initial feature representations. To reduce the channel dimensionality while preserving relevant channel information, we employ an attention-augmented patch descriptor on the extracted features. We then apply differentiable top-k feature adaptation to train the patch descriptor, mapping the extracted feature representations to a new vector space, enabling effective detection of anomalies. Experiments show that ADFA outperforms state-of-the-art (SOTA) methods on multiple challenging medical image datasets, confirming its effectiveness in medical anomaly detection.
BibTeX
@inproceedings{huang2023adfa,
  author = {Yiming Huang and Guole Liu and Yaoru Luo and Ge Yang},
  title = {{ADFA: Attention-augmented Differentiable top-k Feature Adaptation for Unsupervised Medical Anomaly Detection}},
  booktitle = {ICIP},
  year = {2023},
  url = {https://openreview.net/forum?id=iutjilWokq},
}
Figure