TL;DR
ExecGap is a novel adversarial patch attack that disrupts robotic Vision-Language-Action (VLA) models by exploiting the physical limits of the robot's body. Instead of just making the model "guess wrong," ExecGap steers predicted commands toward directions that are kinematically inefficient or dynamically weak, causing the robot to struggle or fail to execute the commanded motion.
What Problem Does This Paper Address?
Vision-Language-Action (VLA) models are the "brains" of modern general-purpose robots, translating visual inputs and text instructions directly into motor commands. While powerful, they are vulnerable to adversarial attacks—specifically visual patches that can trick the model into performing the wrong task.
However, most existing attacks focus solely on the prediction side: making the model output a semantically incorrect command (e.g., "move left" instead of "move right"). The authors of ExecGap argue that this ignores a critical part of the pipeline: the embodiment.
In the real world, a command issued by a VLA model ($u_t$) is not always perfectly realized as motion ($e_t$). Physical constraints like joint limits, singularities (where the robot loses degrees of freedom), and torque limits create a command-execution gap. This paper explores how an attacker can exploit this gap to induce "execution degradation," making the robot physically unable to follow through on its own (seemingly plausible) commands.
Methods
The core of the ExecGap framework is a robot-aware optimization process that identifies "low-executability" directions based on two factors:
- Kinematic Inefficiency: Using the robot's Jacobian matrix (which relates joint speeds to end-effector speeds), the attack identifies directions where moving the joints produces very little actual progress in task space. These are often directions near singularities or joint limits.
- Dynamic Weak Actuation: By analyzing the robot's rigid-body dynamics and torque limits, the attack identifies directions where the robot has the least "acceleration authority." This is visualized as a Feasible Acceleration Polytope; the attack targets the facets of this polytope closest to the origin.
Once these weak directions are identified, the researchers optimize a 50x50 pixel adversarial patch. The optimization objective is to minimize a surrogate loss that encourages the VLA model's output to align with these physically difficult directions. Crucially, this is a white-box attack, meaning the attacker has access to the VLA model's weights and the robot's physical parameters.
Datasets
The authors evaluated ExecGap using:
- LIBERO Suites: A standard benchmark for robotic manipulation containing four suites (Spatial, Object, Goal, and Long), each with 10 distinct tasks.
- Expert Trajectories: Used to provide "clean" data for patch optimization.
- Victim Models: The attack was tested against three prominent VLA architectures: OpenVLA, OpenVLA-OFT, and $\pi_0$.
Key Insights and Ideas
- The Body as an Attack Surface: The most significant insight is that adversarial robustness isn't just a software/vision problem; it's an embodied one. An attack that understands the robot's physics is more insidious than one that just adds noise to an image.
- Execution Degradation vs. Semantic Confusion: Traditional attacks might make a robot drop an object by commanding "open gripper." ExecGap might make the robot fail by commanding a move that requires more torque than the motors can provide, leading to a slow, stuttering failure that might be harder for simple safety monitors to catch.
- Synergy of Kinematics and Dynamics: The authors found that combining both kinematic and dynamic constraints ($\lambda \approx 0.15$) yielded the strongest attacks. Kinematics helps identify "stiff" configurations, while dynamics accounts for the actual "strength" of the actuators.
Key Results
ExecGap consistently outperformed existing state-of-the-art patch attacks (like TMA and UADA) across multiple metrics:
- Task Success Rate (TSR): In simulation, ExecGap reduced the success rate of OpenVLA to 0.0% across all four LIBERO suites.
- Command-Execution Mismatch: ExecGap induced significantly higher CEDE (Command-Execution Direction Error), meaning the robot's actual movement deviated wildly from the model's intended direction.
- Real-Robot Validation: On physical hardware (Kinova Jaco2 and ROKAE xMate ER7 Pro), the attack remained effective. For example, on the Jaco2 arm using OpenVLA, the success rate dropped from 64% to 28% when the physical patch was introduced.
Limitations and Future Work
- White-Box Requirements: Currently, ExecGap requires full access to the VLA model and the robot's physical model (URDF, etc.). Future work could explore "black-box" versions that estimate these vulnerabilities from observation.
- Static Patches: The current attack uses a static patch placed in the environment. Dynamic or "universal" patches that work across many different environments would be a more potent threat.
- Defense: The paper focuses on the attack. The next logical step for the community is developing "execution-aware" VLA models or downstream controllers that can detect and reject commands that fall into these low-executability "traps."
Summary updated