Abstract
Machine Learning as a Service (MLaaS) platforms are increasingly vulnerable to black-box adversarial attacks, where malicious actors iteratively query models to find adversarial perturbations. Current defenses primarily monitor query similarity in the input space, a strategy that can be bypassed by adaptive attackers. This paper introduces Gradient Watch Adversarial Detection (GWAD), a novel framework that shifts the focus from query content to the geometric patterns of query updates. By analyzing Delta Similarity (DS)—the cosine similarity between consecutive updates—GWAD identifies the unique mathematical "fingerprint" left by zeroth-order optimization processes. Our evaluation shows that GWAD achieves over 99% detection accuracy across various datasets and remains robust against sophisticated adaptive strategies that circumvent existing state-of-the-art defenses.
Introduction
The proliferation of Machine Learning as a Service (MLaaS) has made high-performance models accessible via simple APIs. However, this accessibility introduces a significant security risk: black-box adversarial attacks. In these scenarios, an attacker lacks internal knowledge of the model but can craft adversarial examples by observing model outputs in response to a sequence of carefully chosen queries.
Traditional "stateful" defenses attempt to detect these attacks by identifying sequences of highly similar queries. While effective against naive attacks, they are often bypassed by adaptive strategies like OARS, which inject diversity into the query sequence to appear benign. We identify a critical oversight in current defensive paradigms: they focus on what is being queried rather than how the queries are evolving. GWAD addresses this gap by monitoring the optimization trajectory itself.
Methodology: The GWAD Framework
The core innovation of GWAD is the use of Delta Similarity (DS) to characterize query sequences. For a sequence of queries $x_1, x_2, \dots, x_n$, we define the update at step $i$ as $\delta_i = x_i - x_{i-1}$. The Delta Similarity is then calculated as the cosine similarity between consecutive updates:
$$DS_i = \frac{\delta_i \cdot \delta_{i-1}}{\|\delta_i\| \|\delta_{i-1}\|}$$
As shown in the architecture above, GWAD maintains a sliding window of queries (typically 256). For each window, it computes a Histogram of DS (HoDS), which serves as a compact feature vector. A lightweight neural network classifier then analyzes these histograms to distinguish between benign user behavior and active adversarial optimization. To further enhance robustness against "query injection" attacks—where an attacker intersperses benign queries to dilute the attack signal—the system can be augmented with a "Screener" (GWAD+) that filters out unrelated queries using edge detection and binary hashing.
The Geometric Signature of Attacks
The effectiveness of DS stems from the fundamental mathematics of high-dimensional spaces. Most black-box attacks rely on zeroth-order optimization to estimate gradients, a process that involves adding random noise vectors to the input. In high-dimensional spaces, two random vectors are almost certainly orthogonal, meaning their cosine similarity will cluster around zero.
As illustrated in the figure above, benign sequences (such as frames from a video feed) exhibit a wide, unstructured DS distribution. In contrast, various adversarial attacks (NES, HSJA, SimBA, etc.) produce highly structured and distinct DS "fingerprints." These signatures are remarkably consistent across different datasets, such as CIFAR-10 and ImageNet, allowing a GWAD model trained on one dataset to generalize effectively to others.
Experimental Results
We evaluated GWAD against eight state-of-the-art black-box attacks across diverse datasets, including standard benchmarks and real-world scenarios like the Hollywood Heads movie dataset and FLIR ADAS driving scenes.
- Detection Performance: GWAD achieved a near-perfect detection rate of 99.42% across all tested attack methods.
- Robustness to Adaptive Attacks: Against the OARS adaptive attack, which successfully bypasses top-tier defenses like Blacklight and PIHA, GWAD maintained a 100% detection rate.
- False Positive Mitigation: In high-similarity benign datasets (e.g., Hollywood Heads), where traditional defenses often suffer from high false positive rates (up to 26%), GWAD maintained significantly higher precision, with false positive rates as low as 0% in many scenarios.
- Resilience to Injection: With the GWAD+ screener enhancement, the system remained effective even when attackers injected ten times more benign queries than malicious ones to hide their tracks.
Conclusion
GWAD represents a paradigm shift in adversarial defense by focusing on the immutable geometric properties of optimization rather than the mutable content of individual queries. By "watching the gradient" through Delta Similarity, we can detect sophisticated black-box attacks in real-time with unprecedented accuracy and robustness. Future work will explore the application of DS analysis to non-gradient-based search attacks and further optimize the stateful monitoring infrastructure for large-scale MLaaS deployments.
Summary updated