Safety of Embodied Navigation: A Survey

Zixia Wang, Jia Hu, Ronghui MuIJCAI 2025

TL;DR

This survey provides a comprehensive overview of safety concerns in embodied navigation, a critical component of AI agents that interact with the physical world. It categorizes existing attacks, defenses, and evaluation methods, highlighting the urgent need for more robust and reliable systems as embodied AI becomes more prevalent.

What Problem Does This Paper Address?

As Artificial Intelligence (AI) agents become more sophisticated, particularly with the rise of Large Language Models (LLMs), there's a growing interest in "Embodied AI." This refers to AI systems that can perceive, reason, and act within physical or simulated environments. A key application of Embodied AI is embodied navigation, where an AI agent must navigate to a specific target in an unfamiliar setting, much like a robot or a self-driving car.

However, the integration of these systems into real-world applications, especially safety-critical ones, raises significant concerns about their safety and reliability. Embodied navigation agents, often powered by deep neural networks, are vulnerable to various attacks that can lead to incorrect decisions, navigation failures, and potentially harmful outcomes. This paper addresses the critical gap in understanding and mitigating these safety risks by providing a structured survey of the current landscape of attacks, defenses, and evaluation methodologies in embodied navigation.

Methods

This survey adopts a comprehensive approach to understanding the safety of embodied navigation by categorizing and analyzing existing research from three primary perspectives:

  • Attack Strategies: The paper classifies attacks into two main categories:
  • Physical Attacks: These manipulate the physical environment to mislead the agent's perception. Examples include adversarial patches placed on objects or changes in lighting conditions.
  • Model-based Attacks: These exploit vulnerabilities within the AI model itself, particularly LLMs. This includes techniques like "jailbreaking" to bypass safety mechanisms or "backdoor attacks" where hidden triggers cause malicious behavior.
  • Defense Mechanisms: The survey outlines defenses that aim to counter these attacks, broadly categorized as:
  • Physical Defenses: Strategies that address environmental manipulations.
  • Model-based Defenses: Techniques that target vulnerabilities within the AI model's architecture or training.
  • The paper also notes that some defenses integrate elements from both categories.
  • Evaluation Methodologies: The survey examines how the safety and robustness of embodied navigation systems are assessed, focusing on datasets and metrics.

Datasets

The paper discusses several datasets relevant to embodied navigation safety, categorizing them based on whether they are designed for "classic models" or for LLM-based agents.

  • Datasets for Classic Models:
    • A physical attack naturalness dataset using human ratings and gaze data.
  • A photo-based 3D benchmark integrating authentic scenes and objects for realistic navigation.
  • Benchmarks designed for multimodal lifelong navigation, requiring agents to locate targets based on various descriptions.
  • Datasets for LLMs:
    • A benchmark suite for evaluating the task-planning capabilities of LLMs in navigation.
  • A framework for assessing physical risks in embodied AI, generated using LLMs and diffusion models.
  • The paper notes that some benchmarks were not originally designed for embodied navigation but have been adapted.

Key Insights and Ideas

  • Dual Threat Landscape: Embodied navigation faces a dual threat from both physical manipulations of the environment and sophisticated model-based attacks that exploit AI vulnerabilities, particularly in LLMs.
  • LLMs as a Double-Edged Sword: While LLMs enhance embodied AI's understanding and reasoning, they also introduce new attack vectors like jailbreaking and backdoor attacks, necessitating specialized defenses.
  • The Importance of Evaluation: Robust evaluation is crucial for ensuring safety. The survey highlights the need for standardized benchmarks and comprehensive metrics that go beyond simple success rates to capture true safety and reliability.
  • Bridging the Gap: There's a significant need to bridge the gap between research on physical attacks and defenses in general computer vision and the specific challenges of embodied navigation in dynamic, real-world environments.
  • Multimodal Future: As AI systems become increasingly multimodal, existing attack and defense strategies may not directly translate, requiring new research directions tailored to these complex models.

Key Results

The paper is a survey, so it doesn't present new experimental results in the traditional sense. Instead, its key "results" are the comprehensive categorization and analysis of existing research. It identifies:

  • A taxonomy of physical attacks (e.g., 2D/3D patches, adversarial light) and model-based attacks (e.g., RL-based, LLM jailbreaks/backdoors).
  • A classification of defense mechanisms, including physical defenses (e.g., detection and removal) and model-based defenses (e.g., prompt-based aggregation, LLM-specific defenses).
  • An overview of evaluation datasets and metrics (e.g., Success Rate, Success weighted by Path Length) used to assess safety.

The paper implicitly highlights that current research is fragmented, with many studies focusing on specific attack-defense pairs or limited scenarios.

Limitations and Future Work

The authors acknowledge several limitations and propose promising future research directions:

  • Limitations:
    • Current research on embodied navigation safety is relatively limited and fragmented.
  • Many existing defenses are adapted from other domains and may not fully address the unique challenges of embodied navigation.
  • Evaluating LLM-based embodied agents is complex, and existing benchmarks may not fully capture safety concerns.
  • Future Work:
    • Enhancing Robustness in Dynamic Environments: Developing attack and defense strategies that are effective in real-world, dynamic settings, moving beyond proof-of-concept approaches.
  • Expanding Attack Types: Investigating more sophisticated attack methodologies, including white-box attacks and attacks on multimodal models.
  • Robust Defense Strategies: Strengthening defenses for LLMs in embodied navigation, focusing on language understanding, adversarial resistance, and multimodal consistency. Runtime monitoring for dynamic threat detection is also suggested.
  • Reliable Evaluation: Developing more systematic and unified evaluation frameworks that ensure fairness and interpretability across different tasks, potentially incorporating human feedback and leveraging LLMs for automated assessment.
  • Verification Techniques: Establishing systematic approaches to verify embodied navigation systems, quantifying robustness thresholds, and computing theoretical bounds for safety guarantees.

Summary updated

BibTeX
@inproceedings{wang2025safety,
  author = {Zixia Wang and Jia Hu and Ronghui Mu},
  title = {{Safety of Embodied Navigation: A Survey}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2025},
}
Figure