SSMD: Semi-Supervised Medical Image Detection with Adaptive Consistency and Heterogeneous Perturbation

Hong-Yu Zhou, Chengdi Wang, Haofeng Li, Gang Wang, Shu Zhang, Weimin Li, Yizhou Yu

TL;DR

This paper introduces SSMD, a novel semi-supervised medical image detection method that leverages unlabeled data to improve performance. It achieves this through an adaptive consistency cost function that prioritizes object detection over background noise and employs heterogeneous perturbations in both feature and image spaces to enhance robustness.

What Problem Does This Paper Address?

Medical image analysis often faces a scarcity of labeled data, making it challenging to train high-performing deep learning models. While semi-supervised learning has shown promise in classification and segmentation tasks, its application to medical image object detection has been less explored. Existing semi-supervised object detection methods often rely on simple data augmentations and may not effectively handle the specific challenges of medical images, such as a high proportion of background and the need for precise localization of small or overlapping objects. This paper aims to bridge this gap by developing a robust and effective semi-supervised object detector for medical images.

Methods

The core of SSMD lies in its innovative approach to semi-supervised learning for object detection. It utilizes a student-teacher framework, where the teacher model's weights are an exponential moving average of the student's. The key contributions are:

  • Adaptive Consistency Cost: This function regularizes predictions by adaptively scaling the loss based on the confidence of detected proposals. This means the model focuses more on learning from high-confidence object detections and less on background noise, which is crucial for medical images where background often dominates.
  • Heterogeneous Perturbation Strategies: To enhance robustness and feature learning, SSMD introduces two novel perturbation techniques:
  • Noisy Residual Block: This module injects learnable noise into intermediate feature maps, allowing the network to adaptively control noise levels across different channels. This is a feature-space perturbation.
  • Instance-level Adversarial Perturbation: This technique applies adversarial noise to the input image, specifically targeting high-confidence foreground proposals to increase the difficulty of consistency regularization. This is an image-space perturbation.
  • Cutout: Standard cutout augmentation is also employed to further perturb input images.

Datasets

The proposed method is evaluated on two publicly available medical imaging datasets:

  • Nuclei Dataset (2018 Data Science Bowl): This dataset contains nuclei images from various cell types and imaging modalities, used for nuclei detection.
  • DeepLesion Dataset: A large-scale dataset of axial CT slices with annotated lesion instances, used for lesion detection.

Key Insights and Ideas

  • Prioritizing Objects over Background: The adaptive consistency cost is a critical insight, recognizing that in medical imaging, focusing regularization on potential objects (high-confidence proposals) rather than the overwhelming background is more effective for learning.
  • Robustness through Diverse Perturbations: The combination of feature-space (noisy residual block) and image-space (adversarial perturbation, cutout) perturbations is key to building robust representations that generalize well.
  • Instance-level Focus for Detection: Unlike classification or segmentation, detection requires instance-level understanding. The instance-level adversarial perturbation directly addresses this by focusing on individual object proposals.
  • Semi-Supervised Learning's Power in Low-Data Regimes: The paper strongly demonstrates that semi-supervised methods, especially SSMD, significantly outperform supervised baselines when labeled data is scarce, highlighting their importance in medical imaging.

Key Results

SSMD consistently outperforms existing semi-supervised object detection baselines (Pseudo-labeling, CSD, FocalMix) across various labeled ratios and network architectures on both DSB and DeepLesion datasets.

  • Significant Gains in Small Labeled Ratios: SSMD shows a particularly large advantage when labeled data is scarce (e.g., 10% labeled data), outperforming FocalMix by over 3 points on the DSB dataset.
  • State-of-the-Art Performance: The paper reports that SSMD achieves state-of-the-art performance across a wide range of settings. For instance, with 50% labeled data and a ResNet-101 backbone, SSMD achieves 69.4% mAP on DSB and 77.9% 4 FPs on DeepLesion.
  • Ablation Studies Confirm Module Effectiveness: Comprehensive ablation studies demonstrate that each proposed module (adaptive consistency cost, noisy residual block, instance-level adversarial perturbation) contributes to the overall performance improvement. The adaptive consistency cost provides a substantial boost, and the combined heterogeneous perturbations further enhance robustness and accuracy.

Limitations and Future Work

  • Hyperparameter Sensitivity: While the paper explores hyperparameter tuning, the performance can still be sensitive to specific choices, particularly for adversarial perturbation parameters (ξ, ε).
  • Computational Cost: Adversarial perturbation typically involves additional forward and backward passes, increasing computational cost.
  • Future Work: The authors suggest exploring more ways to improve the proposed method, implying potential avenues for further research in enhancing its efficiency and effectiveness.

Summary updated

Abstract

Semi-Supervised classification and segmentation methods have been widely investigated in medical image analysis. Both approaches can improve the performance of fully-supervised methods with additional unlabeled data. However, as a fundamental task, semi-supervised object detection has not gained enough attention in the field of medical image analysis. In this paper, we propose a novel Semi-Supervised Medical image Detector (SSMD). The motivation behind SSMD is to provide free yet effective supervision for unlabeled data, by regularizing the predictions at each position to be consistent. To achieve the above idea, we develop a novel adaptive consistency cost function to regularize different components in the predictions. Moreover, we introduce heterogeneous perturbation strategies that work in both feature space and image space, so that the proposed detector is promising to produce powerful image representations and robust predictions. Extensive experimental results show that the proposed SSMD achieves the state-of-the-art performance at a wide range of settings. We also demonstrate the strength of each proposed module with comprehensive ablation studies.
BibTeX
@article{zhou2021ssmd,
  author = {Hong-Yu Zhou and Chengdi Wang and Haofeng Li and Gang Wang and Shu Zhang and Weimin Li and Yizhou Yu},
  title = {{SSMD: Semi-Supervised Medical Image Detection with Adaptive Consistency and Heterogeneous Perturbation}},
  journal = {Medical Image Analysis},
  year = {2021},
  url = {https://openreview.net/forum?id=CyvYjwAdtXo},
}
Figure