Introduction
As Large Language Models (LLMs) transition from text-based interfaces to physical embodiments—such as drones and robotic assistants—the nature of safety risks shifts from "generative risks" (e.g., offensive text) to "behavioral risks." A malicious instruction in a physical context can lead to property damage, privacy violations, or physical harm. This paper addresses two critical gaps in the current landscape: the lack of safety benchmarks specifically for embodied agents and the challenge of "prompt interference," where complex system instructions confuse existing safety filters.
The EAsafetyBench Taxonomy
To standardize the evaluation of behavioral risks, the authors introduced EAsafetyBench, a taxonomy defining seven categories of harm specific to embodied agents:
- Violent Crimes: Intentional physical injury or killing.
- Non-Violent Crimes: Damage to property.
- Privacy Violations: Unauthorized surveillance in sensitive spaces.
- Malfunction: Forcing the agent into obstacles.
- Discrimination or Hate: Targeting victims based on protected characteristics.
- Harassment: Intentionally causing fear of bodily injury.
- Illegal Assistance: Commanding agents to bypass security measures (e.g., blocking safety cameras).
Pinpoint: Prompt-Decoupled Moderation
The primary technical contribution is Pinpoint, a lightning-fast internal moderation tool designed to detect malicious instructions during the inference phase. Unlike external safety filters that process the entire prompt as a single string, Pinpoint utilizes a "prompt-decoupled" approach to isolate the user's command from the robot's functional system instructions.
Pinpoint operates through three key mechanisms:
- Instruction Localization: Special tokens (
<|begin_of_instruction|>) demarcate the user command. - Masked Attention: At the middle layers of the LLM, a mask is applied to force the model to focus exclusively on the user's instruction, preventing the surrounding "functional prompt" from acting as noise.
- Lightweight Classification: Features extracted from these middle layers are processed by a 3-layer Multi-Layer Perceptron (MLP) with only 4 million parameters to determine the safety of the command.
Experimental Results and Insights
Pinpoint was evaluated across 10 different LLMs (including Llama 3, Mistral, and Qwen) using the EAsafetyBench-Drone dataset, which contains over 9,000 instructions.
The "Middle Layer" Sweet Spot
A key finding of the study is that feature extraction is most effective at the middle layers (e.g., layer 10 to 17 depending on the model). Shallow layers lack sufficient semantic processing, while deeper layers become "contaminated" by the system prompt's influence. By targeting the middle layers, Pinpoint achieved an average detection accuracy of 94.58%.
Efficiency and Real-Time Performance
In physical robotics, latency is a safety-critical metric. Pinpoint demonstrates extreme efficiency, requiring only 0.002 seconds per instance. This is significantly faster than existing baselines like ToxicDetector or GradSafe, which can take hundreds of times longer due to their reliance on external APIs or larger secondary models.
Conclusion and Future Work
Pinpoint provides a robust, real-time solution for securing embodied agents against malicious instructions. While highly effective, the method currently requires "white-box" access to the model's internal hidden states, limiting its use to open-source models deployed on-device. Future research will focus on expanding the taxonomy beyond drone-centric scenarios and developing multi-modal moderation techniques to account for vision and sensor data in complex environments.
Summary updated