Plug-and-Play Feature Generation for Few-Shot Medical Image Classification

Qianyu Guo, Huifang Du, Xing Jia, Shuyong Gao, Yan Teng, Haofen Wang, Wenqiang Zhang

Introduction

In clinical environments, obtaining large-scale labeled datasets is often impractical due to the high cost of expert annotation and the rarity of certain conditions. Few-Shot Learning (FSL) offers a potential solution by enabling models to learn from a limited number of examples. However, applying FSL to medical imaging presents two significant challenges: distribution bias and the semantic gap. Distribution bias occurs when a handful of samples fail to represent the full variance of a disease, leading to overfitting. The semantic gap arises because most FSL models are pre-trained on natural images (e.g., ImageNet), resulting in feature extractors that may overlook subtle, clinically relevant details in X-rays or pathology slides.

To address these issues, we present MedMFG (Medical Meta-Feature Generation), a lightweight, "plug-and-play" module designed to enhance feature representation and expand the available data points for the classifier through synthetic generation.

The MedMFG Framework

MedMFG is designed to be integrated seamlessly behind standard feature extractors (backbones) like ResNet. It consists of three primary components that work in synergy to refine and augment the feature space.

1. Self-construction Feature Module (SFM)

The SFM utilizes self-attention mechanisms, specifically Transformer layers, to identify and weight the most discriminative regions within the few available samples. By focusing on the texture and morphology of lesions rather than irrelevant background noise, the SFM ensures that the extracted features are highly relevant to the diagnostic task.

2. Inter-construction Feature Module (IFM)

The IFM employs cross-attention to compare "support" images (labeled examples) with "query" images (unlabeled test samples). This process helps the model identify shared characteristics across different instances of the same class, effectively narrowing the gap between known and unknown samples.

3. Variational Sample Generation Module (VSGM)

The VSGM is the core engine of the framework. It utilizes an MLP-based Variational Autoencoder (VAE) to take the refined features from the SFM and IFM and generate multiple synthetic variations. By populating the latent space with these plausible feature-level variations, the VSGM provides the classifier with a richer dataset, allowing it to establish more robust decision boundaries even in 1-shot or 5-shot scenarios.

The system is optimized using a multi-part loss function that accounts for reconstruction accuracy, KL divergence for latent space regularization, and final classification performance.

Experimental Evaluation

The efficacy of MedMFG was evaluated across two rigorous cross-domain benchmarks:

  • CDFSL (Cross-Domain Few-Shot Learning): Models were pre-trained on natural images (miniImageNet) and tested on medical datasets including ISIC (skin lesions) and ChestX (chest X-rays).
  • FHIST (Histological Images): Models were pre-trained on colorectal tissues (CRC-TP) and tested on diverse histological datasets such as NCT, LC25000, and BreakHis.

Key Results and Insights

MedMFG consistently achieved state-of-the-art (SOTA) performance across various tasks. Notably, in the CRC-TP to NCT 1-shot classification task, MedMFG reached 69.2% accuracy, a significant improvement over the 64.7% achieved by standard fine-tuning methods.

Key observations from the study include:

  • Feature-Level Augmentation: Unlike traditional image-level augmentation (e.g., rotation, flipping), generating features in the latent space allows for more complex and semantically meaningful variations, which is crucial for bridging the distribution gap in medical data.
  • The Synergy of Attention and Generation: Ablation studies indicated that while attention modules (SFM/IFM) provide some benefit, the most substantial gains occur when they are used to "pre-filter" features before they enter the VSGM. This ensures the generated samples are class-distinctive rather than noisy.
  • Architecture Agnostic: MedMFG demonstrated consistent improvements (ranging from 2.9% to over 10%) when added to different backbones, including ResNet-10, ResNet-18, and Conv-4, without requiring full retraining of the backbone.
  • Improved Class Separation: t-SNE visualizations confirmed that MedMFG helps transform overlapping raw features into tight, well-separated clusters, simplifying the task for the final classifier.

Conclusion and Future Work

MedMFG provides a robust, modular solution for few-shot medical image classification by combining attention-based refinement with variational feature generation. While the current implementation focuses on 2D images, future research will explore extending this framework to 3D modalities such as CT and MRI. Additionally, while MedMFG significantly enhances the classifier's performance, it remains dependent on the initial features captured by the pre-trained backbone; improving the underlying feature extraction for medical domains remains a priority for the field.

Summary updated

BibTeX
@inproceedings{guo2023plug,
  author = {Qianyu Guo and Huifang Du and Xing Jia and Shuyong Gao and Yan Teng and Haofen Wang and Wenqiang Zhang},
  title = {{Plug-and-Play Feature Generation for Few-Shot Medical Image Classification}},
  booktitle = {BIBM},
  year = {2023},
  doi = {10.1109/bibm58861.2023.10385845},
}
Figure