Abstract
Adversarial attacks pose significant threats to the reliability and safety of deep learning models, especially in critical domains such as medical imaging. This paper introduces a novel framework that integrates conformal prediction with game-theoretic defensive strategies to enhance model robustness against both known and unknown adversarial perturbations. We address three primary research questions: constructing valid and efficient conformal prediction sets under known attacks (RQ1), ensuring coverage under unknown attacks through conservative thresholding (RQ2), and determining optimal defensive strategies within a zero-sum game framework (RQ3). Our methodology involves training specialized defensive models against specific attack types and employing maximum and minimum classifiers to aggregate defenses effectively. Extensive experiments conducted on the MedMNIST datasets, including PathMNIST, OrganAMNIST, and TissueMNIST, demonstrate that our approach maintains high coverage guarantees while minimizing prediction set sizes. The game-theoretic analysis reveals that the optimal defensive strategy often converges to a singular robust model, outperforming uniform and simple strategies across all evaluated datasets. This work advances the state-of-the-art in uncertainty quantification and adversarial robustness, providing a reliable mechanism for deploying deep learning models in adversarial environments.
Introduction
Deep learning models in medical imaging are notoriously vulnerable to adversarial attacks—subtle, often imperceptible perturbations to an image that can lead a model to an incorrect diagnosis. While adversarial training offers some defense, there remains a critical need for methods that quantify the reliability of a model's output under attack.
Conformal Prediction (CP) has emerged as a powerful tool for uncertainty quantification, generating a prediction set $\Gamma(x_i)$ that is guaranteed to contain the true label with a user-defined probability (e.g., 90%). However, CP typically relies on the "exchangeability" assumption, which posits that test data follows the same distribution as calibration data. Adversarial attacks violate this assumption, causing prediction sets to either lose their coverage guarantees or become excessively large and clinically useless.
Methodology
To restore statistical guarantees in adversarial settings, we propose a three-tiered framework based on the following notation:
1. Known Attacks (RQ1)
When the specific attack type $g_j$ is known, we restore exchangeability by applying the same attack to the calibration set $\mathcal{I}_{\text{cal}}$. By calculating the quantile threshold $q_{1-\alpha}$ on attacked calibration data, we ensure that the resulting prediction sets are both valid and efficient for that specific threat.
2. Unknown Attacks (RQ2)
In scenarios where the attack type is unknown, we employ Conservative Thresholding. We calculate the required error thresholds for a variety of potential attack types and select the maximum quantile: $q^* = \max_j q_{1-\alpha, j}$. This "safety net" ensures that the coverage guarantee holds for the worst-case scenario among the considered attacks.
3. Strategic Defense (RQ3)
We frame the interaction between the defender and the attacker as a zero-sum game. The attacker seeks to maximize the ambiguity (size) of the prediction set, while the defender seeks to minimize it while maintaining the $1-\alpha$ coverage guarantee. By constructing a payoff matrix of set sizes across different specialized defensive models $f_j$ and attack types $g_j$, we identify the Nash Equilibrium. This equilibrium provides the optimal strategy—either a specific model or a mixture of models—to minimize prediction set size against a strategic adversary.
Experimental Setup
The framework was evaluated using the MedMNIST suite, a collection of standardized biomedical images including PathMNIST (colon pathology), OrganAMNIST (abdominal CT), and TissueMNIST (kidney cortex cells). We utilized ResNet18 as the backbone architecture and compared several CP methods, including Adaptive Prediction Sets (APS), Regularized Selective Conformal Prediction (RSCP), and Vectorized Robust Conformal Prediction (VRCP).
Key Results and Discussion
Guaranteed Coverage and Efficiency
Across all datasets, the framework successfully maintained the target coverage (e.g., 90%) even under aggressive attacks like Projected Gradient Descent (PGD) and Simultaneous Perturbation Stochastic Approximation (SPSA). The game-theoretic "Nash" strategy consistently yielded smaller, more precise prediction sets compared to naive uniform defenses or non-specialized models.
The Value of Specialization
Our findings indicate that training specialized models for specific attacks and strategically selecting between them via game theory outperforms a single "one-size-fits-all" robust model. In high-stakes medical fields, this approach is superior because it prioritizes "honest" uncertainty; it is safer for a model to return a set of possible diagnoses than to provide a confident but incorrect single label.
Methodological Comparisons
Among the CP methods tested, APS generally provided the most stable coverage and the lowest Size-Stratified Coverage Violation. This suggests that APS is particularly robust for maintaining consistent guarantees across different prediction set sizes.
Conclusion
This research demonstrates that combining Conformal Prediction with game theory provides a robust mechanism for medical AI to handle adversarial threats. By treating ambiguity as a payoff and utilizing conservative thresholding, we can provide clinicians with reliable, statistically-backed uncertainty estimates. Future work will focus on dynamic adaptation to novel, unseen attack types and integrating uncertainty-reducing training methods to further refine prediction set efficiency.
Summary updated