Abstract
This paper develops a method for robots to integrate stability into actively seeking out informative measurements through coverage. We derive a controller using hybrid systems theory that allows us to consider safe equilibrium policies during active data collection. We show that our method is able to maintain Lyapunov attractiveness while still actively seeking out data. Using incremental sparse Gaussian processes, we define distributions which allow a robot to actively seek out informative measurements. We illustrate our methods for shape estimation using a cart double pendulum, dynamic model learning of a hovering quadrotor, and generating galloping gaits starting from stationary equilibrium by learning a dynamics model for the half-cheetah system from the Roboschool environment.
Introduction
In robot learning, the quality of the data collected directly dictates the accuracy of the resulting model. While "passive" data collection—observing a robot perform a task—is safe, it often fails to capture the system's behavior in edge cases or high-uncertainty regimes. To learn effectively, robots must engage in active exploration, intentionally moving into unknown states to observe system reactions.
However, active exploration is inherently risky. Pushing a robot into unknown regimes can lead to instability, hardware damage, or catastrophic failure. Existing methods often struggle with a "local minima" problem where the robot stays in safe but uninformative regions, or they fail to provide rigorous safety guarantees during the data collection process itself. This paper bridges the gap between information maximization (learning as much as possible) and Lyapunov stability (ensuring the robot remains safe).
Methodology
The proposed algorithm treats data acquisition as a dynamic area coverage problem. The robot's goal is to cover an "importance distribution"—a spatial map of where new data is most needed—while maintaining a tether to a known stable state.
Hybrid Control Strategy
Rather than using a single controller, the system employs a hybrid strategy that switches between two modes:
- Equilibrium Policy: A known controller that keeps the robot stable and safe (the "home base").
- Exploratory Controller: A policy designed to push the robot into high-uncertainty regions.
To determine when to switch, the authors utilize Mode Insertion Gradients from hybrid systems theory. This technique calculates the sensitivity of a cost function to a brief "insertion" of a new control action. This allows the robot to calculate exactly how to deviate from its safe policy to maximize information gain without losing the ability to recover.
Information Gain and Coverage
The robot minimizes the Kullback-Leibler (KL) divergence between its actual trajectory and a spatial importance distribution. To define this distribution, the system uses Incremental Sparse Gaussian Processes (GPs). The robot identifies "informative" regions based on a measure of linear independence, effectively asking: "Will this new data point provide information that is not already contained in my current dataset?"
Lyapunov Attractiveness
A critical contribution of this work is the mathematical proof of Lyapunov attractiveness. The authors demonstrate that even when the robot performs exploratory "bursts" into unknown state spaces, the hybrid controller ensures the system is guaranteed to eventually return to a stable region of attraction. This ensures that exploration does not lead to "runaway" instability.
Experimental Evaluation
The algorithm was evaluated across three complex simulated environments:
- Cart Double Pendulum: The robot performed shape estimation (sensing a sinusoidal floor) while simultaneously maintaining an upright balance.
- 22-Dimensional Quadrotor: The robot learned a dynamic model while maintaining a stable hover. The method achieved significantly lower $L_2$ modeling error compared to standard "noise injection" (motor babble) because it systematically targeted unexplored regions.
- 26-Dimensional Half-Cheetah (Roboschool): In this locomotion task, the robot learned a dynamics model starting from a stationary equilibrium. The data collected allowed the robot to transition into a forward-moving gallop with higher velocity and lower variance than baseline methods.
Discussion and Key Insights
The research highlights that safety should not just be a constraint on the final task, but a fundamental part of the collection process. By treating a known stable policy as a "home base," the robot can venture into the unknown with a mathematical guarantee of recovery.
Unlike common techniques like "motor babble," which add random Gaussian noise to actuators and lack temporal consistency, this gradient-based approach chooses exploratory actions that are both informative and recoverable. The results show that systematic exploration leads to superior model accuracy and prevents the failures typically associated with pushing a system to its limits.
Limitations and Future Work
While promising, the method currently assumes the existence of an initial (even if approximate) dynamic model and a local stable policy. It is not a "model-free" approach from scratch. Additionally, calculating mode insertion gradients for extremely high-dimensional systems or complex non-linear kernels remains computationally demanding. Future research will focus on scaling these computations and validating the safety guarantees on physical hardware, where sensor noise and unmodeled disturbances present additional challenges.
Summary updated