Introduction
In adversarial machine learning, the focus has shifted from "norm-bounded" attacks—which make infinitesimal pixel changes—to unrestricted adversarial examples (AEs). These attacks modify semantic features, such as color or texture, to bypass defenses. While researchers often claim these modifications are "imperceptible" or "natural," such assertions have historically relied on small-scale, anecdotal evidence.
SCOOTER (Statistical Check Of Open-source Tools for Evaluating Realness) is an open-source framework designed to bring statistical rigor to these claims. By evaluating six state-of-the-art attacks against over 34,000 human ratings, SCOOTER reveals a significant gap between model-fooling capabilities and human perception.
Methodology
The SCOOTER framework employs a three-phase pipeline to ensure data quality and statistical validity.
Phase 1 & 2: Data Collection and Quality Control
To ensure reliable results, participants undergo a multi-stage screening process:
- Preliminary Checks: Includes a colorblindness test (essential for color-based attacks) and a comprehension check to ensure the task is understood.
- Annotation Phase: Participants rate images on a 5-point Likert scale ranging from "Definitely Modified" (-2) to "Definitely Real" (+2). The system incorporates "bogus items" and instruction manipulation checks to filter out inattentive workers.
Phase 3: Statistical Analysis
Rather than simply identifying differences, SCOOTER utilizes Two One-Sided Tests (TOST) for equivalence. This allows researchers to statistically prove whether an adversarial image is practically equivalent to a real one within a predefined margin (±0.2).
The SCOOTER Benchmark
The study introduced ImageNet S-R50-N, a curated subset of 2,966 ImageNet images containing single, clear objects correctly classified by a robust ResNet-50. The full benchmark includes these real images alongside 7,000 adversarial examples generated by six different attacks (SemAdv, cAdv, NCF, DA, AdvPP, ACA), supported by 34,000+ human ratings from 346 participants.
Key Findings
The "Imperceptibility" Myth
The study found that every tested attack failed the test of human equivalence. Regardless of complexity—from simple color shifts to advanced diffusion-based methods—humans could consistently distinguish adversarial images from real ones. Notably, the Natural Color Fool (NCF) attack achieved a 92% success rate against models but was rated as one of the most obviously modified by humans.
Metric Mismatch
A critical insight is the failure of standard objective metrics like Fréchet Distance (FD). These metrics often ranked the "realness" of attacks in the inverse order of human perception, highlighting the necessity of human-in-the-loop evaluation.
AI as a Proxy
The researchers explored using GPT-4o as a preliminary "smoke test." While it lacks human nuance, GPT-4o successfully detected adversarial examples for four out of six attacks, suggesting its utility as a cost-effective initial filter.
Limitations and Future Outlook
While SCOOTER provides a robust framework, it is limited by demographic skews in the participant pool and the lack of environmental control inherent in online studies. Future work aims to leverage the SCOOTER benchmark to develop "human-aligned" objective metrics, potentially automating the evaluation of image authenticity in adversarial contexts.
Summary updated