AFiRe: Anatomy-Driven Self-Supervised Learning for Fine-Grained Representation in Radiographic Images

Yihang Liu, Lianghua He, Ying Wen, Longzhen Yang, Hongzhou ChenAAAI 2025 · DOI

TL;DR

This paper introduces AFiRe, a novel self-supervised learning framework for radiographic images that focuses on anatomical details rather than just global image features. By aligning image tokens with anatomical structures and restoring local anomalies, AFiRe significantly improves fine-grained representation learning, leading to better generalization and anomaly detection in medical imaging tasks.

What Problem Does This Paper Address?

Current self-supervised learning (SSL) methods, particularly those based on contrastive learning, excel at learning general representations from natural images by focusing on global image discrimination. However, these methods often fall short when applied to radiographic images. Radiographs require a much finer level of detail, where subtle anatomical structures and their precise spatial relationships are crucial for accurate diagnosis. The challenge lies in capturing these fine-grained anatomical details effectively without relying on extensive expert annotations, which are costly and time-consuming to obtain. Existing methods often overlook these critical local features, leading to suboptimal performance in medical image analysis tasks.

Methods

AFiRe tackles this challenge by proposing a two-pronged self-supervised approach that leverages the unique token-processing capabilities of Vision Transformers (ViTs):

  • Token-wise Anatomy-Guided Contrastive Learning (Task I): Instead of treating entire images as samples, AFiRe treats each ViT token as an individual sample. This task aligns image tokens based on structural and categorical consistency. It uses "spatial-aware prototypes" to guide the learning process, essentially acting as pseudo-cluster labels for anatomical structures. This helps in learning robust anatomical discrimination by maximizing mutual information between tokens that represent similar anatomical structures.
  • Pixel-level Anomaly-Removal Restoration (Task II): This task focuses on local anomalies. It strategically identifies and removes abnormal tokens within the latent space and then restores them to their normal anatomical counterparts. This process refines the learned discrimination by incorporating detailed geometrical information, crucial for identifying subtle pathologies.
  • Synthetic Lesion Mask (SLM): To enhance anatomical diversity without corrupting intra-consistency (which can happen with traditional augmentations like cropping), AFiRe introduces SLM. This technique dynamically synthesizes various pathological anomalies within a single normal image, creating more realistic and diverse training data.

Datasets

The paper utilizes the following datasets:

  • Pre-training: A large dataset of 811,170 Chest X-ray (CXR) images, comprising 81,117 normal CXR images from MIMIC-CXR-JPG and synthetically generated abnormal images.
  • Evaluation:
    • ChildCXRs: A dataset of healthy and pneumonia-related anomaly images.
  • NIH: A dataset of frontal-view CXR images with 14 disease labels.
  • CXP: A dataset of CXR images with multiple disease labels.
  • SIIM-ARC: Used for segmentation tasks.

Key Insights and Ideas

  • Anatomy is Key: The core insight is that for medical images, focusing on anatomical consistency and fine-grained details is more effective than global image discrimination, especially for self-supervised learning.
  • ViT Tokens as Anatomical Units: The paper leverages the fact that ViT tokens can naturally represent local anatomical structures, allowing for a more granular approach to learning representations.
  • Prototypes for Structure: Spatial-aware prototypes act as a powerful mechanism to guide the contrastive learning towards understanding anatomical structures, rather than just visual similarity.
  • Restoration for Geometry: Pixel-level restoration, specifically targeting anomaly removal, is crucial for preserving and learning precise geometrical information, which is vital for diagnosis.
  • SLM for Realistic Augmentation: The Synthetic Lesion Mask is a clever way to generate diverse and anatomically plausible anomalies, overcoming limitations of traditional data augmentation techniques.

Key Results

  • Robust Anatomical Discrimination: AFiRe demonstrates superior clustering of features compared to state-of-the-art contrastive learning methods, both in terms of anomaly-based and structure-based discrimination.
  • Superior Generalization: AFiRe significantly outperforms 11 other self-supervised methods on multi-label classification tasks across NIH and CXP datasets, especially under limited labeling conditions. For instance, with only 1% labeling, AFiRe achieved the highest mean AUC of 63.2% on NIH.
  • Effective Anomaly Detection: AFiRe shows strong performance in anomaly detection tasks, outperforming recent state-of-the-art methods with significant improvements in AUC, ACC, and F1 scores across ChildCXR, CXP, and NIH datasets. For example, on ChildCXR, AFiRe achieved an ACC of 83.9% and F1 of 87.6%.
  • Visualizations: Grad-CAM visualizations show that AFiRe highlights lesion regions more precisely and focuses on local anatomical details compared to other methods.

Limitations and Future Work

  • Prioritizing Primary Abnormalities: The authors note that AFiRe tends to prioritize primary abnormalities, potentially overlooking larger affected regions.
  • Future Work: The authors suggest improving the model's capability for more accurate detection of lesion regions as a future research direction. They also mention exploring the impact of the cluster number K and the trade-offs between computational resources and performance.

Summary updated

BibTeX
@inproceedings{liu2025afire,
  author = {Yihang Liu and Lianghua He and Ying Wen and Longzhen Yang and Hongzhou Chen},
  title = {{AFiRe: Anatomy-Driven Self-Supervised Learning for Fine-Grained Representation in Radiographic Images}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2025},
  doi = {10.1609/aaai.v39i18.34091},
}
Figure