TL;DR
This paper introduces a novel reinforcement learning (RL) approach to create sparse black-box adversarial attacks on video recognition models. By intelligently selecting key frames and regions to perturb, their method significantly reduces the computational cost and query numbers required for successful attacks, making them more efficient and practical.
What Problem Does This Paper Address?
Deep learning models are increasingly deployed in critical applications, but they are susceptible to adversarial attacks. These attacks involve making subtle, imperceptible changes to input data that cause the model to misclassify it with high confidence. While white-box attacks (where attackers have full model access) are well-studied, black-box attacks (where the model's internal workings are unknown) are more challenging, especially for high-dimensional data like videos.
Previous black-box attacks on videos often involved perturbing every frame (dense attacks), which is computationally expensive. Sparse attacks, which target only specific key frames and regions, offer a more efficient alternative. However, effectively identifying these key elements in a black-box setting remains a significant challenge. Existing methods for sparse attack often rely on heuristic approaches for selecting key frames, which can be time-consuming and less efficient. This paper aims to address this by proposing a more intelligent and efficient method for sparse black-box adversarial attacks on video recognition models.
Methods
The core of this paper's contribution lies in its novel use of Reinforcement Learning (RL) for the crucial task of selecting key frames in a sparse adversarial attack.
- RL-based Key Frame Selection: The authors frame the key frame selection process as a Markov Decision Process (MDP). An RL agent is trained to make decisions about which frames are "unimportant" and can be deleted from the attack's consideration. The agent learns by observing the differences between the original video's class and the target class, receiving rewards from a "threat model" (the video recognition model being attacked) that indicate the quality of its frame selection. This allows the agent to learn an optimized strategy for identifying frames that contribute most to the attack's success.
- Two-Stage Policy Optimization: To further accelerate the learning process, the RL policy is trained in two stages: first, a general policy is learned across various video classes, and then it's fine-tuned on examples from the specific target class. This makes the training more efficient.
- Saliency Detection for Spatial Sparsity: For selecting key regions within frames, the paper employs a fine-grained saliency detection approach. This method leverages the idea that certain regions in an image are more important for recognition, similar to human visual attention.
- Sign-OPT for Gradient Estimation: To boost attack efficiency and reduce query numbers, the method estimates only the sign of the gradient rather than the full gradient itself. This is a technique known as Sign-OPT, which is particularly useful in black-box settings where querying the model is costly.
Datasets
The experiments were conducted on two widely used video datasets:
- UCF-101: Contains 13,320 videos across 101 action categories.
- HMDB-51: Contains 7,000 clips from 51 action classes.
The datasets were split into 70% for training and 30% for testing. For pretraining the RL agent, the training set was used for both clean videos and target class videos. For fine-tuning, a test set video was used as the clean video, and the training set as the target class videos.
Key Insights and Ideas
- RL as an Intelligent Frame Selector: The most significant insight is that RL can learn a sophisticated strategy for identifying frames that are less critical for video recognition, thereby enabling more efficient sparse attacks. This moves beyond simple heuristics.
- Efficiency Gains from Sparsity: The combination of RL-based frame selection and saliency-based region selection demonstrably leads to substantial reductions in query numbers and attack time compared to existing methods.
- Sign-OPT's Role in Query Reduction: Utilizing only the sign of the gradient is crucial for minimizing the number of model queries needed, a key bottleneck in black-box attacks.
- Two-Stage Training for Generalizability: The two-stage RL training approach allows the model to learn general attack principles first, then specialize, leading to better performance and faster adaptation.
Key Results
The paper reports significant improvements in attack efficiency:
- Reduced Query Numbers: The proposed method reduces query numbers by more than 38% for untargeted attacks and more than 34% for targeted attacks compared to existing methods.
- Reduced Attack Time: The average time for finishing an attack is also significantly reduced.
- Improved Sparsity: The method achieves better sparsity (proportion of unperturbed pixels) than heuristic-based methods, indicating more targeted and less visually intrusive attacks.
- Effectiveness: The attacks successfully achieve the desired misclassification (fooling ratio) while staying within defined perturbation bounds (MAP). For instance, the method can reach the MAP bound within a limited iteration number, whereas other methods fail on certain models.
Limitations and Future Work
- Computational Cost of Pretraining: While the attack itself is efficient, the pretraining of the RL agent can be computationally intensive. However, the authors argue that this cost is amortized over many attacks.
- Future Work: The authors suggest investigating black-box attacks on video recognition models that require even fewer queries and less time. This implies further exploration into more efficient gradient estimation techniques or attack strategies.
Summary updated