On Evaluating Adversarial Robustness of Chest X-ray Classification: Pitfalls and Best Practices

Salah Ghamizi, Maxime Cordy, Michail Papadakis, Yves Le TraonarXiv 2022 · OpenReview

Abstract

Vulnerability to adversarial attacks is a well-known weakness of Deep Neural Networks. While most studies focus on natural images with standardized benchmarks like ImageNet and CIFAR, little research has considered real-world applications, particularly in the medical domain. Our research shows that, contrary to previous claims, the robustness of chest X-ray (CXR) classification is much harder to evaluate and leads to very different assessments based on the dataset, architecture, and robustness metric. We argue that previous studies did not take into account the peculiarities of medical diagnosis, such as the co-occurrence of diseases, the disagreement of labelers, the threat model of the attacks, and the risk implications for each successful attack. In this paper, we discuss the methodological foundations, review the pitfalls and best practices, and suggest new methodological considerations for evaluating the robustness of CXR classification models. Our evaluation on 3 datasets, 7 models, and 18 diseases is the largest evaluation of robustness in this field.

Introduction

Deep Learning models for chest X-ray classification have achieved impressive performance, yet they remain vulnerable to adversarial examples—inputs engineered with imperceptible perturbations to trigger misclassification. Current research often treats medical images like "natural" images (e.g., cats or dogs), ignoring the complexities of clinical diagnosis. In reality, medical diagnosis is rarely a binary choice; diseases often co-occur, and the severity of misclassification varies significantly. Previous studies, often limited to single datasets or binary tasks, have claimed near-total vulnerability. This paper establishes a medically-grounded framework to more accurately evaluate the robustness of these models.

Methodology and Datasets

The researchers performed a systematic review of existing literature and conducted the largest robustness evaluation to date. The study utilized three major public datasets:

  • NIH Chest X-ray14: 112,000 images with automated labels.
  • CheXpert: 224,000 images labeled via NLP on radiology reports.
  • PadChest: 160,000 images, with 27% manually annotated by physicians.
  • AllD: A combined benchmark of the above datasets to test large-scale generalization.

Two key innovations were introduced:

  1. k-robust accuracy: Instead of checking only the single top predicted label, this metric evaluates whether the top k predicted labels intersect with the top k ground-truth labels. This accounts for the multi-label nature of CXR diagnosis and labeler uncertainty.
  2. RISK Metric & Targeted Attacks: A threat model based on a co-occurrence matrix was developed. Attacks were designed to force "improbable" disease combinations (e.g., conditions that rarely occur together). The RISK metric quantifies the clinical danger of these adversarial outputs.

Key Insights

The Multi-label Complexity

Evaluating robustness on binary "Normal vs. Disease" tasks overestimates vulnerability. Multi-label classification is inherently more complex to attack meaningfully. Furthermore, standard metrics like Accuracy or AUC fail to distinguish between harmless errors and catastrophic clinical risks.

Dataset and Architecture Dependency

Robustness is highly dependent on the training data. A model might appear robust on its own training distribution but fail significantly when evaluated on a different dataset. For instance, the NIH-trained model demonstrated higher inherent robustness compared to the CheXpert model.

Top-k accuracy for transfer attacks across different source and target datasets.

Results

The evaluation revealed several critical findings:

  • Context-Dependent Vulnerability: Under a standard PGD attack ($\epsilon = 1/255$), the NIH-trained model maintained an average 1-robust accuracy of approximately 15.4% across various transfer attack sources.
  • Resilience in Top-k Predictions: Models appeared significantly stronger when evaluated using 3-robust accuracy. For example, the AllD model's robustness reached an average of 35.6%, suggesting that correct labels often remain within the top three predictions even under attack.
  • Lethality of Risk-Based Attacks: Targeted attacks using the RISK metric were far more effective. The NIH model’s robust accuracy dropped from 13.78% in an untargeted setting to just 2.69% when specifically targeted.
Evaluation of robustness metrics and the RISK metric across different loss functions.

Conclusion and Future Work

The study emphasizes that there is no "one-size-fits-all" metric for robustness in medical AI. Future research should focus on:

  • Developing adversarial defenses tailored for multi-label medical data.
  • Creating evaluation protocols that mirror clinical use cases, such as human-in-the-loop systems.
  • Investigating the architectural factors that contribute to resilience across diverse datasets.

Summary updated

Abstract

Vulnerability to adversarial attacks is a well-known weakness of Deep Neural Networks. While most of the studies focus on natural images with standardized benchmarks like ImageNet and CIFAR, little research has considered real world applications, in particular in the medical domain. Our research shows that, contrary to previous claims, robustness of chest x-ray classification is much harder to evaluate and leads to very different assessments based on the dataset, the architecture and robustness metric. We argue that previous studies did not take into account the peculiarity of medical diagnosis, like the co-occurrence of diseases, the disagreement of labellers (domain experts), the threat model of the attacks and the risk implications for each successful attack. In this paper, we discuss the methodological foundations, review the pitfalls and best practices, and suggest new methodological considerations for evaluating the robustness of chest xray classification models. Our evaluation on 3 datasets, 7 models, and 18 diseases is the largest evaluation of robustness of chest x-ray classification models.
BibTeX
@misc{ghamizi2022evaluating,
  author = {Salah Ghamizi and Maxime Cordy and Michail Papadakis and Yves Le Traon},
  title = {{On Evaluating Adversarial Robustness of Chest X-ray Classification: Pitfalls and Best Practices}},
  year = {2022},
  eprint = {2212.08130},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2212.08130},
}
Figure