Abstract
Humanoid robots controlled by Reinforcement Learning (RL) offer high performance but lack formal safety guarantees. We present SHIELD, a safety framework that bridges this gap by combining generative machine learning with stochastic control theory. SHIELD functions as a plug-and-play safety layer that modulates reference velocities to ensure obstacle avoidance in probability. By learning system uncertainty via a Conditional Variational Autoencoder (CVAE) and employing Stochastic Discrete-Time Control Barrier Functions (S-DTCBF), SHIELD provides a robust safety filter for "black-box" controllers. Experimental results on the Unitree G1 humanoid demonstrate successful navigation in complex environments with improved tracking and tunable risk levels.
1. Introduction
The deployment of humanoid robots in human-centric environments requires rigorous safety guarantees. While Reinforcement Learning (RL) has enabled sophisticated locomotion and handling of complex dynamics, these policies are often "black boxes" that lack the transparency required for formal safety proofs. Traditional methods like Control Barrier Functions (CBFs) provide mathematical safety guarantees but are highly sensitive to model inaccuracies—a significant hurdle for complex, multi-jointed humanoids operating on uneven ground.
This paper introduces SHIELD, a framework designed to enforce safety on complex robots without requiring a perfect physics model or retraining existing controllers. SHIELD treats the underlying RL policy as part of the system dynamics and applies a safety filter at the reference signal level, preserving the robot's gait while ensuring collision avoidance.
2. Methodology
SHIELD employs a three-layered approach to bridge the gap between high-level commands and low-level safety.
2.1 Constraint Specification
Safety is defined using a "safe set" within a simplified reduced-order model. By abstracting the 23-degree-of-freedom robot into a simpler representation (such as a single integrator), the framework can efficiently compute safety boundaries, such as maintaining a minimum distance (e.g., 0.5 meters) from obstacles.
2.2 Dynamics Residual Learning
To account for the discrepancy between the simplified model and the actual hardware, SHIELD utilizes a Conditional Variational Autoencoder (CVAE). The CVAE is trained on real-world rollout data to learn the "residual"—the difference between the commanded velocity and the actual movement. This captures hardware noise, communication delays, and the specific quirks of the underlying RL policy.
2.3 Safety-Aware Reference Generation
The core of SHIELD is the Stochastic Discrete-Time Control Barrier Function (S-DTCBF). Instead of overriding joint torques directly, which can disrupt delicate walking gaits, SHIELD modulates the user's desired velocity. It uses the learned CVAE model and Freedman’s Inequality to bound the risk of safety violations. This ensures the robot remains safe with a high, user-defined probability ($P$).
3. Experimental Results
The framework was validated through both simulation and real-world hardware experiments.
3.1 Hardware Implementation
SHIELD was deployed on a Unitree G1 humanoid robot. Training data consisted of approximately 6 minutes of hardware rollouts, capturing state, command, and disturbance tuples. The system utilized onboard LiDAR and a Jetson Orin NX for real-time computation, demonstrating the efficiency of the framework.
3.2 Performance and Navigation
In navigation tasks, SHIELD successfully guided the G1 through crowds of moving pedestrians and static obstacles. Key findings include:
- Improved Tracking: The CVAE-based compensation allowed the robot to track intended velocities more accurately than baseline controllers by predicting and compensating for disturbances.
- Tunable Conservatism: The risk parameter ($P$) acts as a "knob" for engineers, allowing for a direct trade-off between safety (staying further from people) and performance (moving faster in tight spaces).
- Comparison to Baselines: Standard CBFs that did not account for learned dynamics often failed due to model mismatch, resulting in either collisions or overly conservative behavior (taking twice as long to reach goals). SHIELD consistently identified the "Goldilocks" zone of safety and efficiency.
4. Discussion and Future Work
SHIELD demonstrates that safety can be retrofitted onto high-performance RL policies without retraining. However, some limitations remain. The current theoretical guarantees are strongest for concave safety functions and static obstacles. Future work will focus on extending these proofs to non-concave multi-obstacle scenarios and integrating semantic perception to allow for more nuanced interactions with different types of obstacles (e.g., distinguishing between a wall and a human).
Summary updated