Positive Safety Margins from Artificial Potential Field Reward Augmentation for Size-Robust Obstacle Avoidance under Partial Observability

Ziyi HaoarXiv 2026 · arXiv

In the world of robotic manipulation, "skimming the surface" is a recipe for disaster. Most reinforcement learning (RL) agents trained for obstacle avoidance learn to be just barely safe enough—they find the shortest path that doesn't result in a collision. However, if an obstacle is slightly larger than expected or its position drifts, that "zero-margin" policy immediately fails.

A new paper by Ziyi Hao from The Chinese University of Hong Kong, titled "Positive Safety Margins from Artificial Potential Field Reward Augmentation for Size-Robust Obstacle Avoidance under Partial Observability," proposes a simple yet effective solution: bake a safety buffer directly into the reward function using Artificial Potential Fields (APF).

TL;DR

By augmenting the reward function with an analytic artificial potential field, RL agents learn to maintain a significant safety margin around obstacles they cannot even "see." This induced buffer makes the resulting policies robust to out-of-distribution obstacle sizes, maintaining high success rates where vanilla RL agents completely fail.

What Problem Does This Paper Address?

In real-world industrial settings, a robot might not always have a clear view of its surroundings due to occlusions or sensor noise. The authors study a "hidden observation" protocol where the robot knows the general layout of its workspace (learned implicitly through training) but does not receive real-time data on the exact size or instantaneous drift of obstacles.

The fundamental issue is structural: if an agent is only penalized for collisions and also pays a "step cost" (to encourage speed), the optimal policy will naturally converge to a trajectory that skims the obstacle's surface. This leaves no room for error. If the obstacle radius increases at test time (an out-of-distribution or OOD scenario), the robot, unaware of the change, will attempt its usual path and collide.

Methods

The researchers used a simulated Franka Emika Panda arm in a reaching task. The environment, PandaReach-v3, requires the end-effector to reach a goal while avoiding two cylindrical pillars.

The core innovation is Reward Augmentation. Instead of just a binary collision penalty, they added a horizontal Artificial Potential Field (APF) term to the reward:

  • The APF Term: It creates a quadratic penalty zone (a "safety margin") extending outward from the obstacle surface.
  • State-Dependent Cost: Unlike "potential-based reward shaping," which is designed to preserve the original optimal policy, this augmentation is a state-dependent cost intended to change the optimal policy, forcing it to favor clearance over the absolute shortest path.
The training loop shows how obstacle information reaches the policy only through the reward signal, as the pillars are withheld from the state observation.

The agent was trained using Proximal Policy Optimization (PPO) under "hidden observation"—meaning the pillar positions and radii were absent from the input state $s$.

Datasets

The study was conducted entirely in simulation using:

  • PyBullet with the panda-gym library.
  • Evaluation Protocol: The agents were trained with a pillar radius of 0.03 m and then evaluated against a "radius sweep" up to 0.055 m to test robustness to OOD obstacle sizes.

Key Insights and Ideas

  • Implicit Layout Learning: Even though the obstacles are "hidden" from the observation, the agent learns the nominal layout of the workcell through its neural network weights. It "knows" where the pillars usually are.
  • Fixed Buffer vs. Adaptation: Because the agent never observes the radius, it doesn't "adapt" to larger obstacles. Instead, the APF reward induces a fixed buffer. This buffer (measured at ~47.9 mm) is large enough to absorb the maximum radius increase (25 mm) tested in the study.
  • The "Safety Tax": Safety isn't free. The robust policies resulted in paths approximately 19% longer than the "skimming" policies.
  • Sample Efficiency: Surprisingly, the APF term didn't just make the robot safer; it made it learn faster. Policies with APF eliminated collisions during training at least eight times earlier (in median steps) than those without.

Key Results

The quantitative gap between the APF-augmented policies and vanilla RL is striking:

  • Safe Success Rate: At the maximum tested radius (0.055 m), APF policies maintained a safe success rate above 88%, while vanilla policies collapsed to below 10%.
  • Minimum Clearance: The APF term induced a minimum clearance of 47.9 mm, compared to just 9.6 mm for the vanilla baseline.
  • Algorithm Agnostic: The benefit persisted across different RL solvers, including PPO, SAC (Soft Actor-Critic), and TD3 (Twin Delayed DDPG).
  • Comparison to Classical Planners: The learned policy's success rate was comparable to a classical Khatib-type APF planner, even though the classical planner had full knowledge of the obstacle positions and the RL policy had none.

Limitations and Future Work

The authors identify several areas for further exploration:

  1. Simulation Gap: The study uses kinematic control and non-terminal collisions (the arm can pass through pillars during training). Real-world deployment would require accounting for dynamics and terminal physical collisions.
  2. Fixed Layouts: The method relies on the obstacles having a "nominal" position. It may not generalize to environments where obstacles move to entirely new, unlearned locations.
  3. Geometric Complexity: The current potential field is designed for simple cylinders. Extending this to arbitrary 3D shapes would require more complex signed distance functions.
  4. Safe RL Baselines: While compared against reward ablations, the authors suggest future work should compare this approach against formal Safe RL methods like Constrained Policy Optimization (CPO) or Lagrangian methods.

Summary updated

Abstract

Collision-free reaching is central to manipulation, yet real cells often leave the size and exact position of an obstacle unobserved, which excludes reinforcement learning controllers that take obstacle geometry as input. A reward limited to a collision penalty does not depend on the clearance outside the obstacle, so under a step cost its learned policy skims the obstacle surface and fails once the obstacle radius exceeds its training value. We investigate whether a single analytic reward term can induce a safety margin without exposing the obstacle to the policy. We augment the reward of a simulated Franka Emika Panda reaching task with a horizontal artificial potential field that penalizes proximity within a safety margin around each of two pillars (fixed nominal layout, random drift). Under hidden observation, policies trained with this term maintain a safe success rate above 88% across a radius sweep from 0.03 m to 0.055 m (30 seeds), while policies without it fall below 10% at the maximum tested radius; the term induces a minimum clearance of 47.9 mm against 9.6 mm without it and eliminates training collisions at least eight times earlier (median steps). The advantage is statistically significant at all out-of-distribution radii and holds across three reinforcement learning solvers and under full observation. Its safe success rate is comparable to that of a classical potential-field planner given the pillar positions. The margin is a fixed buffer rather than an adaptation to the radius, and it lengthens the path by about 19%.
BibTeX
@misc{hao2026positive,
  author = {Ziyi Hao},
  title = {{Positive Safety Margins from Artificial Potential Field Reward Augmentation for Size-Robust Obstacle Avoidance under Partial Observability}},
  year = {2026},
  eprint = {2611.02947},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.02947},
}
Figure