Abstract
Controlling underactuated robotic systems requires balancing computational efficiency with physical robustness. This paper introduces Zero Dynamics Policies (ZDP), a framework that integrates the formal stability of control theory with the representational power of machine learning. By learning a low-dimensional manifold that is invariant under optimal control actions, we enable a 3D hopping robot to execute agile maneuvers—such as climbing stairs and crossing narrow bridges—with high robustness and minimal online computational overhead.
Introduction
The control of underactuated systems, where the number of control inputs is less than the degrees of freedom, remains a fundamental challenge in robotics. Legged locomotion is a primary example, requiring precise coordination of actuated joints to influence unactuated global dynamics (e.g., the center of mass). Existing approaches typically fall into three categories: Model Predictive Control (MPC), which is computationally intensive; Reinforcement Learning (RL), which often lacks physical transparency; and heuristics, which may lack formal stability guarantees.
This work proposes a "middle ground" that exploits the physical structure of underactuation. By focusing on the zero dynamics of the system, we can achieve the performance of optimal control with the execution speed of simple heuristics.
Methodology: Zero Dynamics Policies
The core of the ZDP framework involves decomposing the robot's state into actuated coordinates ($\eta$) and unactuated coordinates ($z$). The objective is to learn a mapping, $\psi_\theta(z)$, that defines the target configuration for the actuated joints based on the current unactuated state.
The Zero Dynamics Manifold
We define a manifold $\mathcal{M}_\psi$ where the actuated states are constrained as a function of the unactuated ones. Stability of the entire system is guaranteed if two conditions are met:
- Invariance: If the robot starts on the manifold and takes an optimal control step, it must remain on the manifold.
- Stability: The "zero dynamics"—the motion of the system while restricted to the manifold—must be stable.
Learning via Optimal Control
Rather than manual tuning, the manifold is learned by training a neural network to approximate trajectories generated by Iterative LQR (iLQR). The network is optimized to represent a manifold that is invariant under these optimal actions. This allows the controller to "remember" optimal behaviors, enabling 1kHz control loops on standard hardware without solving complex optimization problems in real-time.
Experimental Validation
The ZDP framework was validated using the ARCHER 3D hopping robot. Training data was generated synthetically by sampling the unactuated state space and solving iLQR problems. The resulting policy was evaluated through over 3,000 physical hops across diverse and challenging terrains.
Key Results
The robot demonstrated exceptional agility and robustness in several scenarios:
- Disturbance Rejection: ARCHER maintained stability while hopping on a treadmill at 1 mph, even when subjected to external shoves.
- Terrain Agility: The robot successfully climbed 1.5-inch stairs and descended 20° ramps without the aid of vision sensors.
- Precision Maneuvering: The system exhibited sufficient accuracy to traverse a narrow 2x4 wooden bridge.
Discussion and Future Work
While ZDPs offer a robust and efficient control solution, some challenges remain. The training process is computationally expensive due to the need for repeated iLQR queries. Additionally, the solver currently requires a stabilizing initial guess (e.g., from a Raibert heuristic) to converge. Future research will explore integrating this structured approach with Reinforcement Learning to handle more complex environments and extending the framework to multi-legged platforms like quadrupeds.
Conclusion
Zero Dynamics Policies provide a powerful framework for achieving robust agility in underactuated robots. By embedding optimal control behavior into a learned, stable manifold, we achieve high-performance locomotion that is both computationally efficient and physically grounded.
Summary updated