Optimal Zero-Shot Detector for Multi-Armed Attacks

Federica Granese, Marco Romanelli, Pablo Piantanida

Abstract

This paper explores a scenario in which a malicious actor employs a multi-armed attack strategy to manipulate data samples, offering them various avenues to introduce noise into the dataset. Our central objective is to protect the data by detecting any alterations to the input. We approach this defensive strategy with utmost caution, operating in an environment where the defender possesses significantly less information compared to the attacker. Specifically, the defender is unable to utilize any data samples for training a defense model or verifying the integrity of the channel. Instead, the defender relies exclusively on a set of pre-existing detectors readily available "off the shelf". To tackle this challenge, we derive an innovative information-theoretic defense approach that optimally aggregates the decisions made by these detectors, eliminating the need for any training data. We further explore a practical use-case scenario for empirical evaluation, where the attacker possesses a pre-trained classifier and launches well-known adversarial attacks against it. Our experiments highlight the effectiveness of our proposed solution, even in scenarios that deviate from the optimal setup.

Introduction

Adversarial attacks—subtle, often imperceptible perturbations designed to mislead machine learning models—pose a significant threat to the deployment of AI in security-critical domains. Traditional defensive strategies typically fall into two categories: robust training, which attempts to harden the model itself, or adversarial detection, which seeks to identify and reject malicious inputs. However, most existing methods rely on a critical assumption: that the defender has access to representative training data or prior knowledge of the specific attack strategy.

In real-world scenarios, this assumption rarely holds. Attackers are dynamic, often employing "multi-armed" strategies where they can pivot between different loss functions, perturbation constraints, and optimization techniques. This paper addresses the challenge of the "Cautious Defender"—an agent who must protect a system without the luxury of retraining or access to a labeled dataset of adversarial examples.

Problem Formulation: The Multi-Armed Attack

We consider a scenario where an attacker can choose from a set of $M$ possible attack strategies. The defender, meanwhile, has access to a pool of $K$ pre-existing, "off-the-shelf" detectors. These detectors might be specialized for specific types of noise or trained on different domains, making them individually "weak" or narrow in scope.

The defender's goal is to aggregate the outputs of these detectors to determine whether an incoming sample is clean or adversarial. The constraint is strictly "zero-shot": the defender cannot see any examples of the attacks beforehand and cannot verify the reliability of the communication channel between the input source and the classifier.

Methodology: Information-Theoretic Aggregation

To solve this, we frame the defense as a minimax problem. We seek an aggregation rule that minimizes the average worst-case regret, ensuring robust performance regardless of which strategy the attacker selects.

Optimal Soft-Detector

Instead of a simple majority vote or linear averaging, we derive an optimal "soft-detector." This detector is a weighted mixture of the available experts. The weights are not static; they are determined by maximizing the Shannon mutual information between the potential attack strategies and the detectors' collective predictions.

The Blahut-Arimoto Algorithm

To compute these optimal weights efficiently for each incoming sample or batch, we utilize the Blahut-Arimoto algorithm. Originally developed in information theory to calculate the capacity of a communication channel, we adapt it here to find the optimal distribution of "trust" across our detector pool. This allows the system to dynamically prioritize the detectors most likely to be effective against the current, unknown attack.

Domain Shift and Theoretical Bounds

One of the primary risks in zero-shot detection is the appearance of a completely novel attack strategy. We provide a theoretical analysis of this "domain shift," establishing an upper bound on detection error based on the statistical distance between the known detector capabilities and the novel attack's characteristics.

Empirical Evaluation

We evaluated the framework using two benchmark computer vision datasets: CIFAR10 and the Street View House Numbers (SVHN) dataset. We employed a ResNet-18 as the target classifier and subjected it to a diverse array of attacks, including:

  • Gradient-based attacks: PGD, FGSM, and BIM.
  • Optimization-based attacks: Carlini-Wagner and DeepFool.
  • Black-box attacks: Square Attack and HopSkipJump.

Key Results

Our experiments demonstrate that the proposed information-theoretic aggregator significantly outperforms state-of-the-art (SOTA) baselines:

  • Performance Gains: On CIFAR10, our method achieved an AUROC improvement of up to 79.5 percentage points over Natural Scene Statistics (NSS) in complex multi-attack environments.
  • Error Reduction: We observed a drastic reduction in the False Positive Rate (FPR) at a 95% True Positive Rate, with improvements reaching 90.3 percentage points in specific $L_\infty$ norm settings.
  • The Power of Redundancy: The results confirm that even "shallow" or simple 3-layer neural network detectors, when aggregated optimally, can rival or exceed the performance of much more complex, specialized detection systems.

Discussion and Insights

The success of this framework highlights two major insights. First, zero-shot defense is viable. By leveraging the mathematical structure of information theory, we can coordinate existing tools to handle novel threats without new data. Second, redundancy acts as a shield. While an attacker might easily bypass a single detector, bypassing an optimized ensemble of diverse detectors is exponentially more difficult.

Limitations and Future Work

While effective, the current approach assumes that at least one detector in the pool has some sensitivity to the attack being used. If an attacker develops a truly "alien" strategy that falls outside the collective expertise of the pool, performance will degrade. Additionally, while the Blahut-Arimoto algorithm is computationally efficient, the requirement to run it per-sample may introduce latency in high-throughput environments. Future research will focus on scaling this framework to larger datasets like ImageNet and investigating defenses against "adaptive attackers" who specifically target the aggregation logic itself.

Summary updated

Abstract

This paper explores a scenario in which a malicious actor employs a multi-armed attack strategy to manipulate data samples, offering them various avenues to introduce noise into the dataset. Our central objective is to protect the data by detecting any alterations to the input. We approach this defensive strategy with utmost caution, operating in an environment where the defender possesses significantly less information compared to the attacker. Specifically, the defender is unable to utilize any data samples for training a defense model or verifying the integrity of the channel. Instead, the defender relies exclusively on a set of pre-existing detectors readily available "off the shelf". To tackle this challenge, we derive an innovative information-theoretic defense approach that optimally aggregates the decisions made by these detectors, eliminating the need for any training data. We further explore a practical use-case scenario for empirical evaluation, where the attacker possesses a pre-trained classifier and launches well-known adversarial attacks against it. Our experiments highlight the effectiveness of our proposed solution, even in scenarios that deviate from the optimal setup.
BibTeX
@inproceedings{granese2024optimal,
  author = {Federica Granese and Marco Romanelli and Pablo Piantanida},
  title = {{Optimal Zero-Shot Detector for Multi-Armed Attacks}},
  booktitle = {International Conference on Artificial Intelligence and Statistics},
  year = {2024},
  url = {https://openreview.net/forum?id=VVsUDNXPCZ},
}
Figure