Introduction
Deep neural networks (DNNs) have revolutionized object detection but remain vulnerable to adversarial perturbations—subtle pixel changes that lead to misclassification. To counter this, "context-aware" detectors have been developed. These systems evaluate the logical consistency of a scene; for instance, a "boat" appearing on a "crosswalk" next to a "stop sign" is flagged as an anomaly. While effective against naive attacks, these defenses introduce a new attack surface.
The Challenge of Contextual Consistency
Traditional black-box attacks often rely on multiple queries to a victim model to refine perturbations. However, high query volumes are easily detected by traffic monitoring. The core challenge is to craft a "zero-query" attack that is both effective and "context-consistent," ensuring the final perturbed scene appears logical to a context-aware defense mechanism.
As shown in Figure 1, a context-inconsistent attack might turn a "crosswalk" into a "boat," which is easily detected. A context-consistent attack, however, holistically alters the scene (e.g., turning the "crosswalk" into "water" and the "man" into a "boat") to maintain logical coherence.
Methodology: The ZQA-PSPM Framework
The researchers propose the Zero-Query Attack (ZQA) framework, which shifts the focus from attacking individual objects to attacking entire scenes.
- Context Modeling: A co-occurrence graph is constructed from training data (e.g., Pascal VOC) to define which object labels logically belong together.
- Perturbation Success Probability Matrix (PSPM): This is the framework's key innovation. Using a surrogate model, the attacker pre-calculates the likelihood of successful label swaps (e.g., dog to cat) under specific perturbation budgets.
- Holistic Attack Planning: Instead of random changes, the algorithm identifies a target object and uses the PSPM and Context Graph to select "bystander" objects to perturb. This ensures the final set of labels forms a consistent subgraph in the context model.
- Execution: The attack is implemented using Projected Gradient Descent (PGD), guided by the holistic plan to ensure the entire scene is transformed convincingly.
Experimental Evaluation
The ZQA-PSPM approach was evaluated on Pascal VOC 2007 and MS COCO benchmarks, focusing on images with 2–6 objects to ensure context was a significant factor.
Key Results
The proposed method demonstrated significant improvements over existing baselines.
As detailed in Figure 6, the ZQA-PSPM approach consistently achieved higher fooling rates than context-agnostic methods across various perturbation budgets ($\epsilon$). Notably, the zero-query attack often outperformed traditional "few-query" attacks that were allowed up to three rounds of feedback from the victim model. Furthermore, the attacks showed strong transferability across different architectures, such as from Faster R-CNN to RetinaNet or FoveaBox.
Conclusion and Future Directions
This research highlights a critical vulnerability: context-aware defenses can be bypassed without any direct interaction with the victim model. While the current approach relies on co-occurrence, future work could incorporate spatial relationships and relative object sizes to create even more sophisticated "lies." This work serves as a call to action for developing more robust, multi-modal defensive strategies in computer vision.
Summary updated