LLM robot planners now commonly split the work among several parts: a language model proposes a program with free numeric parameters, and a physics simulator searches for values that satisfy collision and stability constraints. We conjecture a specific weakness in this design. The simulator computes in double precision and accepts any plan that satisfies the constraints, including plans that clear an obstacle by a fraction of a millimetre. A real robot cannot place, sense, or repeat motions at that resolution. We call these knife-edge plans. If they are common, they would fail from ordinary sensor and actuator noise, which is a safety problem. They would also give an attacker an easy target, which is a security problem: a spoofed pose reading kept below the sensor's noise floor, or a harmless-sounding instruction such as "pack them as tightly as possible", could push the planner toward plans that pass simulation and then fail on hardware.
The proposal is to measure how often this happens and to close the gap with one quantity. Each plan gets a margin, meaning how far its execution parameters and observed scene can be perturbed before a constraint breaks. This margin is compared with a tolerance $\epsilon$ taken from the physical limits of the actual sensors and actuators. The same $\epsilon$ serves as the safety noise model and as the attacker's budget. A plan is executed only if it survives worst-case perturbation inside that budget, not merely a random draw from it. The measured margin is also returned to the planning agent as structured feedback, so the planner learns which constraints are tight.
Core Claim
Our claim is that simulator-verified LLM plans hold a meaningful share of their constraint satisfaction below the physical resolution of the hardware. If so, a single worst-case margin test at a physically calibrated $\epsilon$ should predict both noise-induced failures and adversarial ones. We also expect it to cut both at a smaller cost in task success than asking a language-model critic to "check safety" or than averaging over random noise. Both parts are testable. The first is a measurement that could come back negative. The second is a comparison against strong alternatives, and one of those alternatives may turn out to be nearly as good.
The physical-limits direction supplies the lens rather than the mechanism. Nayebi's critique of hypercomputation (Practical Intractability; Plausible hypercomputability) argues that idealized models hide "embedded infinities" such as infinite precision. It also argues that a physical computation must be usable by a finite observer: its states must be readable, settable, and repeatable within finite precision. We turn those qualitative criteria into a concrete number. Readability becomes a bound on how precisely the robot can measure a pose. Settability becomes a bound on how precisely it can place one. Repeatability becomes a bound on how much repeated executions vary. Together these define $\epsilon$. We do not rely on the stronger Turing-versus-real-number computability argument of Böck et al., because it has not been shown to bear on floating-point robot planning. We cite it only as motivation: digital approximations of continuous dynamics hide implementation choices.
Research Program
Pipeline. The testbed is the PRoC3S pipeline (Trust the PRoC3S). There, an LLM writes a language-model program, meaning code with open continuous parameters, and a PyBullet sampler finds values that satisfy the constraints. We also run InstructFlow (InstructFlow, code at chiht21/InstructFlow). In InstructFlow, a separate constraint-generator agent turns simulator failures into symbolic predicates such as ClearOf(gripper, obstacle), and a planner uses them to repair its plan. These systems supply the method we build on. Our addition is a verifier inside the simulator, not another LLM agent.
Margin. For an accepted plan with parameters $\theta$ and observed scene $s$, the margin is
$$m(\theta, s) = \min_{\|\delta_\theta\| \le \epsilon_{set},\ \|\delta_s\| \le \epsilon_{read}} \; \min_k g_k(\theta + \delta_\theta, s + \delta_s).$$
Here $g_k$ is the signed satisfaction of constraint $k$, such as clearance distance or a stability residual. It is positive when the constraint holds. The terms $\epsilon_{set}$ and $\epsilon_{read}$ are the settability and readability tolerances. A plan is executed only if $m > 0$.
In the Drawing domain the inner minimum is close to analytic. In the block and YCB object domains, where constraints come from contact physics, we estimate it with an adversarial search: random restarts followed by CMA-ES inside the $\epsilon$-ball. This gives an upper bound on the true margin, so it catches real vulnerabilities but does not certify robustness. We report it as exactly that. The tolerances are calibrated from published UR5 and Panda repeatability specifications and from pose-estimation noise we measure ourselves, and we sweep them across a range.
Feedback. When a plan fails the margin test, the verifier returns the tight constraint $k^\star$ and the direction of the perturbation that broke it. It does this in the same symbolic vocabulary InstructFlow already uses. Repair is then targeted at the constraint that failed.
Two attack channels. Both are aimed at the joints between agents that task decomposition creates. The first is sub-noise-floor perturbation of the numeric scene description passed from perception to planner, limited to $\epsilon_{read}$. The second is text injection into the task instruction that preserves the goal but pressures the planner toward tight tolerances. The second connects the text-channel weaknesses documented in Can a Large Language Model Keep My Secrets? to physical consequences. The safety-security convergence literature motivates this threat model. That literature documents cyber-to-physical causal chains (Curran et al.; Lesi et al.) and shows that physical coupling bypasses digital policy boundaries (IoTSafe). None of that work studies LLM planners whose own output is the attack surface.
Baselines
Naive reference. Code-as-Policies-style direct generation, with no sampler and no margin test. PRoC3S reports this family below 40% success on its constrained tasks.
Strongest existing method. PRoC3S itself. It reports 70% to 90% success on Drawing and Arrange-Blocks. We also run InstructFlow, which reports a 20–40% success improvement over PRoC3S, LLM3, and Code-as-Policies. It also reports only an 11–16% success drop under up to 50% object-size noise and corrupted feedback.
Both are the right competitors because they already ground plans in physics and already handle some noise. If knife-edge plans are rare for them, our premise fails. No reported result was found on Lacuna for any of these planners under bounded adversarial perturbation or tolerance-pressuring instructions, so we establish those numbers ourselves.
Fair comparison. All methods use the same LLM, the same task seeds, and the same prompt budget. They also share the same total simulator-call budget. Calls our verifier spends on adversarial search are charged against the sampler's budget, so any gain cannot come from extra compute.
Ablations. Three ablations isolate the key ingredient. The first replaces the worst-case minimum with the average over random draws from the same ball, which is domain randomization at equal cost. The second keeps the margin gate but removes the structured feedback to the planner. The third replaces the verifier with an LLM critic agent that is told the tolerance. The critic runs as a Tester-style role following Self-collaboration Code Generation.
What counts as winning. The method beats the baselines if, at matched budget, it lowers attack success under both channels by a large amount while keeping clean-task success within a few points of InstructFlow.
Why not just…?
Why not just use domain randomization, sampling noise during constraint solving? This is the real threat, and the proposal is built around testing it. Averaging over random noise is cheap and may remove most knife-edge plans that fail under ordinary noise. What it cannot do is target the thin directions an attacker searches for. A plan can survive 99% of random draws and still fail at a specific perturbation well inside the budget. The worst-case-versus-average ablation settles this directly. If randomization matches worst-case gating on adversarial attack success at equal simulator budget, we will report that the simpler method suffices and that the contribution is the measurement and the threat model.
Why not just add an LLM safety critic agent? Critics break confirmation bias in code generation (Self-collaboration). But judging a sub-millimetre margin requires computing it, not reasoning about it in text. Monolithic models also show a large gap between being useful and recognizing physical danger: Defining and Evaluating Physical Safety for LLMs reports GPT-3.5 at 93.75% utility but 12.5% self-assurance. A text-reading critic also inherits exposure to the injection channel. The critic ablation measures both weaknesses.
Why not just put a control barrier function filter at execution time? A control barrier function is a certificate that keeps a system inside a safe set, and a filter based on one corrects commands online. Zhong et al. extend this to enforce safety and opacity together. The approach needs polynomial dynamics and handles set-invariance constraints, not the contact-rich stability constraints in block stacking. It also acts only after a doomed plan has been chosen, and it cannot fix a stack that is unstable once placed. Where it applies, in obstacle clearance for Drawing and for drones, we include it as a complementary layer rather than a competitor.
Why not just use a larger model? Scale does not change the fact that the simulator accepts any feasible plan, however thin its margin. Knife-edge plans come from the sampler's acceptance rule, not from the LLM's skill. We test this by repeating the prevalence measurement with two model sizes.
Proposed Experiments
Experiment 1: prevalence of knife-edge plans. On Drawing, Arrange-Blocks, and Arrange-YCB, we record the distribution of margins for plans that PRoC3S and InstructFlow accept. For each $\epsilon$ in the sweep, we report the fraction of plans with $m \le 0$. A small fraction at realistic $\epsilon$ falsifies the premise.
Experiment 2: does the margin predict failure? We measure the area under the ROC curve (AUROC) of the margin for predicting execution failure under two perturbation types: held-out random noise and a separately run adversary. The AUROC compares the margin with a simple predictor based on minimum clearance. The claim that one quantity covers both safety and security requires high AUROC on both.
Experiment 3: the main comparison. At matched simulator budget we measure clean success, success under random perturbation at $\epsilon$, and attack success rate for each channel. This is run for every baseline and ablation above.
Experiment 4: transfer to drones. We transfer the method to the AirSim drone benchmark of Defining and Evaluating Physical Safety for LLMs. There, clearance margins are analytic and collision avoidance is already reported high: CodeLlama-7B-Instruct reaches 99.12%. The question is whether that figure holds once waypoints and obstacle readings are perturbed within GPS and inertial-sensor tolerances.
Optional hardware check. A small set of plans run on a physical arm, split into near-zero and high-margin groups, would test whether margin predicts real-world failure. This is the most decisive check if hardware is available.
References
- Practical Intractability: A Critique of the Hypercomputation Movement.
- Plausible hypercomputability
- Computing-Model and Computing-Hardware Selection for ICT Under Societal and Judicial Constraints
- Trust the PRoC3S: Solving Long-Horizon Robotics Problems with LLMs and Constraint Satisfaction
- InstructFlow: Adaptive Symbolic Constraint-Guided Code Generation for Long-Horizon Planning
- Can a Large Language Model Keep My Secrets? A Study on LLM-Controlled Agents
- Achieving the Safety and Security of the End-to-End AV Pipeline
- Security Analysis for Distributed IoT-Based Industrial Automation
- IoTSafe: Enforcing Safety and Security Policy with Real IoT Physical Interaction Discovery
- Self-Collaboration Code Generation via ChatGPT
- Defining and Evaluating Physical Safety for Large Language Models
- Secure-by-Construction Controller Synthesis via Control Barrier Functions