Towards Reliable Code-as-Policies: A Neuro-Symbolic Framework for Embodied Task Planning

Sanghyun Ahn, Wonje Choi, Junyong Lee, Jinwoo Park, Honguk Woo

TL;DR

While Large Language Models (LLMs) excel at generating robot control code via the "Code-as-Policies" paradigm, they often fail in real-world, partially observable environments due to a lack of environmental grounding. To solve this, the authors introduce NESYRO, a neuro-symbolic framework that combines static symbolic verification with interactive runtime validation. When the robot encounters environmental uncertainty, it recursively generates safe exploratory code to gather missing information, drastically improving task success rates.

What Problem Does This Paper Address?

The "Code-as-Policies" paradigm allows LLMs to directly write executable code (like Python scripts) to guide robotic actions, translating high-level natural language instructions into low-level motor skills. This works beautifully in fully observable, pristine simulation environments. However, put that same robot in a messy, dynamic, or partially observable real-world setting, and things quickly fall apart.

If an LLM doesn't know whether a drawer is locked, or lacks precise depth information for a fragile object, it might generate code that blindly attempts an action, leading to catastrophic failures or irreversible damage. Existing frameworks lack a mechanism to say, "I don't know enough about the environment yet; let me safely check first." NESYRO bridges this gap by enabling robots to explicitly reason about uncertainty and verify code correctness before execution.

Figure 1: Concept of our NESYRO framework illustrated with an example of a room-cleaning task where drawer states are initially unknown. While (1) naive code generation fails without detecting missing observations, (2) NESYRO recursively probes the environment to recover drawer states, enabling the generation of grounded code that successfully completes the task.

Methods

NESYRO (Neuro-Symbolic Robot task planning) splits the code generation process into a recursive two-phase pipeline:

  1. Neuro-Symbolic Code Verification: The LLM generates a task specification alongside the initial policy code. A symbolic verification tool (an SMT solver, specifically Z3) statically checks the code for logical consistency against the specification. If it fails, the solver provides detailed feedback to the LLM for iterative refinement.
  2. Neuro-Symbolic Code Validation: Once verified, the code is broken down into sequential skills. Each skill is evaluated using a combined Neuro-Symbolic Confidence score ($NeSyConf$), which multiplies Common Sense Confidence (CSC, from a validation LLM) and Logic Confidence (LC, from a PDDL planner).
Figure 2: The NESYRO framework with Neuro-Symbolic Code Verification and Neuro-Symbolic Code Validation phases. It recursively verifies and validates the policy code, while incrementally acquiring observations.

If the confidence score drops below a set threshold, NESYRO triggers a Safe Probe. The system generates exploratory code to interact with the environment and recover the missing state (e.g., checking if a drawer is locked) without disrupting the overall task. This creates a recursive policy tree where exploratory subroutines ground the parent code before final execution.

Datasets

The authors evaluated NESYRO across both simulation and physical hardware:

  • Simulation: They utilized RLBench, a standard benchmark for tabletop robotic manipulation. They tested across four task categories: Object Relocation, Object Interaction, Auxiliary Manipulation, and Long-Horizon tasks.
  • Real-World: Experiments were conducted using a physical 7-DoF Franka Emika Research 3 robotic arm equipped with an Intel RealSense D435 RGB-D camera.

To rigorously test partial observability, they introduced four levels of environmental incompleteness: High, Low, Stochastic, and Complete.

Key Insights and Ideas

  • Static Verification is Only Half the Battle: Verifying that code is syntactically and logically sound (via SMT solvers) doesn't mean it will work in the real world. Embodied agents require runtime validation that actively probes physical realities.
  • Uncertainty as a Subroutine: Instead of failing and completely replanning from scratch when an unexpected state occurs, NESYRO treats exploration as a safe, temporary detour (a probe) that feeds data back into the main program.
  • The Neuro-Symbolic Sweet Spot: LLMs provide the common-sense intuition needed to figure out how to explore (e.g., "if it's dark, I should turn on the light"), while symbolic tools (PDDL/SMT) enforce strict logical constraints, preventing the LLM from hallucinating impossible or dangerous actions.

Key Results

NESYRO demonstrated massive improvements over standard LLM code-generation baselines:

  • In RLBench simulations, it achieved a 46.2% improvement in task success rate over the state-of-the-art Code-as-Policies baseline.
  • In real-world testing under partial observability, NESYRO achieved over 86.8% executability of task-relevant actions.
  • It significantly reduced Irreversible Actions (IA)—instances where the robot causes damage or renders a task uncompletable—dropping the total real-world error count from 29 (using the CodeSift baseline) down to just 7.

Limitations and Future Work

While powerful, the current iteration of NESYRO relies on a binary Logic Confidence metric and predefined domain knowledge, which can restrict its adaptability to entirely novel environments. The authors note that future work will focus on integrating probabilistic and temporal reasoning frameworks, such as probabilistic PDDL, to handle more complex real-world nuances. Additionally, they plan to expand the framework so it can validate skills that aren't explicitly predefined in its initial domain knowledge.

Summary updated

Abstract

Recent advances in large language models (LLMs) have enabled the automatic generation of executable code for task planning and control in embodied agents such as robots, demonstrating the potential of LLM-based embodied intelligence. However, these LLM-based code-as-policies approaches often suffer from limited environmental grounding, particularly in dynamic or partially observable settings, leading to suboptimal task success rates due to incorrect or incomplete code generation. In this work, we propose a neuro-symbolic embodied task planning framework that incorporates explicit symbolic verification and interactive validation processes during code generation. In the validation phase, the framework generates exploratory code that actively interacts with the environment to acquire missing observations while preserving task-relevant states. This integrated process enhances the grounding of generated code, resulting in improved task reliability and success rates in complex environments. We evaluate our framework on RLBench and in real-world settings across dynamic, partially observable scenarios. Experimental results demonstrate that our framework improves task success rates by 46.2% over Code-as-Policies baselines and attains over 86.8% executability of task-relevant actions, thereby enhancing the reliability of task planning in dynamic environments.
BibTeX
@inproceedings{ahn2025towards,
  author = {Sanghyun Ahn and Wonje Choi and Junyong Lee and Jinwoo Park and Honguk Woo},
  title = {{Towards Reliable Code-as-Policies: A Neuro-Symbolic Framework for Embodied Task Planning}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2025},
  url = {https://openreview.net/forum?id=VaC4sa96EI},
}
Figure