Abstract
Visual Object Tracking (VOT) is a fundamental component in real-time autonomous systems, such as drones and self-driving vehicles. Unlike static image classification, VOT requires continuous, frame-by-frame predictions, making it susceptible to unique adversarial challenges. This paper presents SPARK (Spatial-aware online incremental attack), a novel framework designed to hijack the trajectory of real-time visual trackers. By leveraging incremental optimization and spatial-temporal sparsity, SPARK generates imperceptible perturbations that can force a tracker to follow a maliciously pre-defined path. Our extensive evaluation across four major benchmarks—OTB100, VOT2018, UAV123, and LaSOT—demonstrates that SPARK is both highly effective and computationally efficient, achieving real-time performance.
Introduction
Adversarial attacks have traditionally focused on static image classification, where subtle perturbations can mislead a model into misidentifying an object. However, real-world applications increasingly rely on Visual Object Tracking (VOT), which operates in an online, real-time manner. Attacking VOT systems is significantly more complex due to two primary hurdles:
- Temporal Transferability: Perturbations must remain effective as the object and background evolve across frames.
- Efficiency: Trackers often operate at 30+ FPS, requiring attacks to be generated with minimal latency.
To address these challenges, we introduce SPARK, an online incremental attack that exploits the temporal continuity of video to efficiently hijack tracker trajectories.
Methodology: The SPARK Framework
SPARK is built upon three core technical pillars designed for speed and stealth:
1. Incremental Optimization
Traditional adversarial attacks often restart the optimization process for every new frame, which is computationally prohibitive for real-time tracking. SPARK utilizes a "warm start" approach by calculating only the incremental difference needed from the previous frame's perturbation. Since consecutive video frames are highly similar, the perturbation required for frame $t$ is likely very close to that of frame $t-1$. This method allows the attack to converge in as few as two iterations, whereas standard iterative attacks might require ten or more, drastically reducing computational overhead.
2. Spatial-Temporal Sparsity
To ensure the attack remains imperceptible to human observers, SPARK employs an $L_{2,1}$ norm constraint. This mathematical regularizer encourages the noise to be spatially sparse—concentrated in small, critical regions—and temporally consistent. This minimizes visual flickering and makes the adversarial changes nearly invisible to the human eye.
3. Trajectory Hijacking
SPARK supports both Untargeted Attacks (UA), which aim to cause the tracker to lose the target, and Targeted Attacks (TA). In a TA scenario, the attacker can specify an arbitrary "Targeted Trajectory" (e.g., a spiral or a straight line away from the target), and SPARK will generate the necessary perturbations to "drive" the tracker along that path.
Experimental Evaluation
We evaluated SPARK against state-of-the-art Siamese-network-based trackers (e.g., SiamRPN, SiamRPN++, SiamDW) using four benchmark datasets: OTB100, VOT2018, UAV123, and LaSOT.
Key Results
- High Success Rate: On the OTB100 benchmark, SPARK achieved a 78.9% success rate in targeted attacks against the SiamRPN-AlexNet tracker. This demonstrates that even sophisticated trackers can be reliably misled by carefully crafted incremental noise.
- Real-Time Efficiency: The incremental optimization allowed SPARK to reach its goal in an average of 2.25 iterations per frame. This efficiency is crucial for real-world applications where the attack must keep pace with the high-speed processing of modern trackers.
- Stealthiness: SPARK produced significantly lower Mean Absolute Perturbation compared to standard iterative attacks like BIM or C&W, maintaining high visual quality and making detection difficult for human observers.
- Cross-Model Transferability: Perturbations generated for shallower models (e.g., AlexNet) were often effective against deeper architectures (e.g., ResNet-50), indicating a shared vulnerability in Siamese tracking paradigms.
Discussion and Limitations
While SPARK is highly effective, its performance varies with model depth. Deeper architectures like ResNet-50 exhibit greater robustness, leading to a decrease in attack success rates. Furthermore, while untargeted attacks transfer well across different models, targeted attacks—which require precise trajectory control—show lower transferability (approximately 6.5%). This suggests that the specific noise patterns required for path-following are highly optimized for the target model's internal representations.
Conclusion
SPARK demonstrates that the temporal continuity of video, while a strength for tracking, is a significant vulnerability that can be exploited for efficient adversarial attacks. By hijacking trajectories in real-time with imperceptible noise, SPARK highlights the need for more robust defensive mechanisms in safety-critical visual tracking applications.
Summary updated