TL;DR
This paper introduces a novel learning framework that combines neural networks with Control Lyapunov Functions (CLFs) to achieve certifiably stable, time- and fuel-optimal guidance for spacecraft rendezvous. By jointly learning a stable Lyapunov function and an optimal control policy, the method ensures safety while generating near-optimal trajectories, even for complex fuel-optimal problems.
What Problem Does This Paper Address?
Spacecraft rendezvous, crucial for missions like deep space exploration and on-orbit servicing, requires precise control for bringing two spacecraft into close proximity. While machine learning (ML) offers promise for autonomous and optimal guidance, traditional "black-box" ML methods often lack formal stability guarantees, which are paramount for safety-critical aerospace applications. Existing methods for optimal control are often computationally expensive, making them unsuitable for onboard implementation. This paper aims to bridge this gap by developing an ML-based guidance system that is both optimal and provably stable, addressing the need for autonomous, safe, and efficient rendezvous maneuvers.
Methods
The core of this work is a comprehensive framework that integrates Control Lyapunov Functions (CLFs) with supervised learning. Key methodological contributions include:
- Neural Candidate Lyapunov Function: A novel neural network is designed to directly output a candidate Lyapunov function that inherently satisfies the positive definiteness property. This simplifies the training process by eliminating the need for explicit constraints on this property.
- Minimizing Lyapunov Derivative: The control policy's thrust direction is determined by minimizing the time derivative of the Lyapunov function. This ensures that the system state moves towards the equilibrium point.
- Minimal Required Throttle: To satisfy the decay condition of the CLF, the thrust throttle is determined using a "minimal required throttle" concept, which replaces traditional loss terms and ensures the decay condition is met.
- Joint Supervision Loss Function: A single, effective loss function is proposed to jointly train both the Lyapunov function and the control policy. This is achieved by leveraging optimal state-control pairs generated offline.
- State-Dependent Decay Rate: The decay rate ($\gamma$) in the CLF is treated as a state-dependent parameter, which was found to significantly improve training efficiency and performance compared to a constant decay rate.
Datasets
The paper generates its own datasets for training and validation.
- Optimal Trajectory Generation: An efficient polynomial mapping method is used to rapidly generate a large number of optimal trajectories. This method involves solving Two-Point Boundary-Value Problems (TPBVPs) and then using differential algebraic techniques to create polynomial maps that can generate new optimal trajectories for randomized initial states.
- Dataset Split: Three datasets are created:
- Training Set (Dtrain): 10,000,000 samples.
- Validation Set (Dval): 100,000 samples.
- Test Set (Dtest): 1,000 samples. Each sample consists of a state and the corresponding optimal control. For the fuel-optimal problem, time-to-go is also included.
Key Insights and Ideas
- Bridging Stability and Optimality: The framework's strength lies in its unified approach to learning both stability certificates (Lyapunov functions) and optimal control policies simultaneously, avoiding the complexities of separate optimization problems.
- Neural Lyapunov Functions Simplify Design: By designing a neural Lyapunov function that inherently satisfies positive definiteness, the authors bypass a common challenge in learning-based stability control, simplifying parameter tuning.
- State-Dependent Decay Rate is Crucial: Treating the decay rate ($\gamma$) as state-dependent significantly enhances training speed and stability, particularly for the fuel-optimal problem where constant $\gamma$ hindered convergence.
- Minimal Required Throttle as a Smart Control Primitive: This concept effectively enforces the CLF decay condition without needing to explicitly penalize deviations from it in the loss function.
Key Results
- Certifiably Stable Guidance: The proposed method generates guidance policies that are proven to be stable through the use of CLFs.
- Near-Optimal Performance: The learned policies achieve near-optimal performance for both time-optimal and fuel-optimal rendezvous problems, closely matching results from traditional optimization methods.
- Time-Optimal: Achieved a final state error of [-0.4 m, 6.3 m, -0.0009 m/s, -0.0144 m/s] for a non-nominal initial condition, very close to the target state.
- Fuel-Optimal: Achieved a fuel consumption penalty of only 0.3779% compared to the optimal value obtained by indirect shooting.
- Robustness: The method demonstrated robustness to initial condition perturbations, with 121 out of 200 cases successfully achieving rendezvous within defined error bounds for the fuel-optimal problem.
- Computational Efficiency: The learned guidance policy can be updated rapidly (3.6 seconds per update) and is computationally efficient, making it suitable for onboard implementation.
Limitations and Future Work
- Fuel-Optimal Performance Gap: The authors acknowledge that the guidance performance for the fuel-optimal problem is not as effective as for the time-optimal problem. This is attributed to the inherent sensitivity of the fixed-final-time fuel-optimal problem to initial state perturbations and the need to replicate both thrust direction and throttle.
- Chattering Phenomenon: The thrust direction control can exhibit chattering, especially as the spacecraft approaches the target. A two-stage rendezvous strategy is suggested to mitigate this.
- Future Work: The authors suggest focusing on enhancing the guidance performance for the fuel-optimal problem. Further research could explore more advanced techniques to address the sensitivity to perturbations and improve the replication of complex throttle profiles.
Summary updated