Attacking CNNs in Histopathology with SNAP: Sporadic and Naturalistic Adversarial Patches (Student Abstract)

Daya Kumar, Abhijith Sharma, Apurva NarayanAAAI 2024 · OpenReview

Abstract

Convolutional neural networks (CNNs) are being increasingly adopted in medical imaging. However, in the race for developing accurate models, their robustness is often overlooked. This elicits a significant concern given the safety-critical nature of the healthcare system. Here, we highlight the vulnerability of CNNs against a sporadic and naturalistic adversarial patch attack (SNAP). We train SNAP to mislead the ResNet50 model predicting metastasis in histopathological scans of lymph node sections, lowering the accuracy by 27%. This work emphasizes the need for defense strategies before deploying CNNs in critical healthcare settings.

Introduction

As deep learning models, particularly Convolutional Neural Networks (CNNs), become integral to diagnostic workflows in digital pathology, their security and reliability are paramount. While these models excel at identifying metastatic cancer in tissue scans, they remain susceptible to adversarial attacks—carefully crafted perturbations designed to induce misclassification.

Traditional adversarial research often focuses on "digital" attacks (global, imperceptible noise) or "patch" attacks (localized, visible stickers). However, standard patch attacks typically utilize large, contiguous squares that are easily detectable by human observers or automated defense algorithms. This study introduces SNAP (Sporadic and Naturalistic Adversarial Patches), a stealthy attack framework designed to mimic biological textures, thereby bypassing traditional defenses while maintaining high efficacy in misleading clinical AI.

Methodology: The SNAP Framework

The SNAP attack is characterized by two innovative design principles: sporadic distribution and naturalistic appearance. Unlike conventional patches, SNAP distributes adversarial perturbations across the image using a binary mask ($m$), creating a "multi-patch" effect that resembles scattered cellular structures rather than artificial artifacts.

Overview of the SNAP framework, illustrating the sporadic patch application and iterative optimization process.

The attack is mathematically formulated as:

$$x' = (1 - m) \odot x + m \odot \delta$$

where $x$ represents the original histopathology image, $m$ is the sporadic mask, and $\delta$ is the learned adversarial patch.

To ensure the attack remains inconspicuous to pathologists, the authors constrained the color space of $\delta$. Instead of random initialization, the patch begins with the median pixel value of the dataset—a specific shade of pink characteristic of H&E-stained slides. The patch is then iteratively optimized to maximize the model's prediction error (specifically targeting false negatives) while remaining within a realistic color range. This collaborative training of multiple small patches allows for the emergence of complementary patterns that are more potent than a single localized patch.

Experimental Setup

The framework was evaluated using the Histopathologic Cancer Detection (HCD) dataset, consisting of 64x64 pixel images of lymph node sections. The primary task was to identify metastatic tissue within the central 32x32 region. For the experiment, images were upscaled to 224x224 to accommodate standard CNN architectures. The attack was trained on 8,000 images and validated on a separate set of 2,000 images.

Key Results and Discussion

The impact of SNAP was tested against a ResNet50 model, which served as a representative workhorse for medical imaging tasks.

  • Baseline Accuracy: The clean, unattacked model achieved a high accuracy of 96.28%.
  • Adversarial Impact: Upon applying the SNAP perturbations, the model's accuracy dropped significantly to 69.0%, a 27% absolute decrease.
  • Clinical Risk: The attack was optimized to produce "Class 0" (no metastasis) predictions. In a clinical setting, such false negatives are critical, as they could result in missed diagnoses and delayed treatment for cancer patients.

The success of SNAP suggests that current defenses, which often rely on detecting contiguous anomalous regions, may be inadequate against sporadic, distributed attacks that blend into the natural background of the image.

Conclusion and Future Directions

This research demonstrates a significant vulnerability in medical CNNs to stealthy, naturalistic adversarial attacks. By mimicking the visual characteristics of biological tissue, SNAP poses a realistic threat to AI-assisted diagnostics.

Future work will involve testing the transferability of SNAP to other architectures, such as Vision Transformers (ViTs), and utilizing Generative Adversarial Networks (GANs) to further refine the naturalistic quality of the patches. Ultimately, this work serves as a "red teaming" exercise, highlighting the urgent need for robust defense mechanisms tailored to sporadic multi-patch attacks before AI is fully integrated into safety-critical healthcare environments.

Summary updated

Abstract

Convolutional neural networks (CNNs) are being increasingly adopted in medical imaging. However, in the race for developing accurate models, their robustness is often overlooked. This elicits a significant concern given the safety-critical nature of the healthcare system. Here, we highlight the vulnerability of CNNs against a sporadic and naturalistic adversarial patch attack (SNAP). We train SNAP to mislead the ResNet50 model predicting metastasis in histopathological scans of lymph node sections, lowering the accuracy by 27%. This work emphasizes the need for defense strategies before deploying CNNs in critical healthcare settings.
BibTeX
@inproceedings{kumar2024attacking,
  author = {Daya Kumar and Abhijith Sharma and Apurva Narayan},
  title = {{Attacking CNNs in Histopathology with SNAP: Sporadic and Naturalistic Adversarial Patches (Student Abstract)}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2024},
  url = {https://openreview.net/forum?id=V3nSrYenyz},
}
Figure