Learning-Based Stable Optimal Guidance for Spacecraft Close-Proximity Operations

Kun Wang, Roberto Armellin, Adam Evans, Harry Holt, Zheng ChenarXiv 2025 · OpenReview

TL;DR

This paper introduces a novel learning framework that combines neural networks with Control Lyapunov Functions (CLFs) to achieve certifiably stable, time- and fuel-optimal guidance for spacecraft rendezvous. By jointly learning a stable Lyapunov function and an optimal control policy, the method ensures safety while generating near-optimal trajectories, even for complex fuel-optimal problems.

What Problem Does This Paper Address?

Spacecraft rendezvous, crucial for missions like deep space exploration and on-orbit servicing, requires precise control for bringing two spacecraft into close proximity. While machine learning (ML) offers promise for autonomous and optimal guidance, traditional "black-box" ML methods often lack formal stability guarantees, which are paramount for safety-critical aerospace applications. Existing methods for optimal control are often computationally expensive, making them unsuitable for onboard implementation. This paper aims to bridge this gap by developing an ML-based guidance system that is both optimal and provably stable, addressing the need for autonomous, safe, and efficient rendezvous maneuvers.

Methods

The core of this work is a comprehensive framework that integrates Control Lyapunov Functions (CLFs) with supervised learning. Key methodological contributions include:

  • Neural Candidate Lyapunov Function: A novel neural network is designed to directly output a candidate Lyapunov function that inherently satisfies the positive definiteness property. This simplifies the training process by eliminating the need for explicit constraints on this property.
  • Minimizing Lyapunov Derivative: The control policy's thrust direction is determined by minimizing the time derivative of the Lyapunov function. This ensures that the system state moves towards the equilibrium point.
  • Minimal Required Throttle: To satisfy the decay condition of the CLF, the thrust throttle is determined using a "minimal required throttle" concept, which replaces traditional loss terms and ensures the decay condition is met.
  • Joint Supervision Loss Function: A single, effective loss function is proposed to jointly train both the Lyapunov function and the control policy. This is achieved by leveraging optimal state-control pairs generated offline.
  • State-Dependent Decay Rate: The decay rate ($\gamma$) in the CLF is treated as a state-dependent parameter, which was found to significantly improve training efficiency and performance compared to a constant decay rate.

Datasets

The paper generates its own datasets for training and validation.

  • Optimal Trajectory Generation: An efficient polynomial mapping method is used to rapidly generate a large number of optimal trajectories. This method involves solving Two-Point Boundary-Value Problems (TPBVPs) and then using differential algebraic techniques to create polynomial maps that can generate new optimal trajectories for randomized initial states.
  • Dataset Split: Three datasets are created:
  • Training Set (Dtrain): 10,000,000 samples.
  • Validation Set (Dval): 100,000 samples.
  • Test Set (Dtest): 1,000 samples. Each sample consists of a state and the corresponding optimal control. For the fuel-optimal problem, time-to-go is also included.

Key Insights and Ideas

  • Bridging Stability and Optimality: The framework's strength lies in its unified approach to learning both stability certificates (Lyapunov functions) and optimal control policies simultaneously, avoiding the complexities of separate optimization problems.
  • Neural Lyapunov Functions Simplify Design: By designing a neural Lyapunov function that inherently satisfies positive definiteness, the authors bypass a common challenge in learning-based stability control, simplifying parameter tuning.
  • State-Dependent Decay Rate is Crucial: Treating the decay rate ($\gamma$) as state-dependent significantly enhances training speed and stability, particularly for the fuel-optimal problem where constant $\gamma$ hindered convergence.
  • Minimal Required Throttle as a Smart Control Primitive: This concept effectively enforces the CLF decay condition without needing to explicitly penalize deviations from it in the loss function.

Key Results

  • Certifiably Stable Guidance: The proposed method generates guidance policies that are proven to be stable through the use of CLFs.
  • Near-Optimal Performance: The learned policies achieve near-optimal performance for both time-optimal and fuel-optimal rendezvous problems, closely matching results from traditional optimization methods.
  • Time-Optimal: Achieved a final state error of [-0.4 m, 6.3 m, -0.0009 m/s, -0.0144 m/s] for a non-nominal initial condition, very close to the target state.
  • Fuel-Optimal: Achieved a fuel consumption penalty of only 0.3779% compared to the optimal value obtained by indirect shooting.
  • Robustness: The method demonstrated robustness to initial condition perturbations, with 121 out of 200 cases successfully achieving rendezvous within defined error bounds for the fuel-optimal problem.
  • Computational Efficiency: The learned guidance policy can be updated rapidly (3.6 seconds per update) and is computationally efficient, making it suitable for onboard implementation.

Limitations and Future Work

  • Fuel-Optimal Performance Gap: The authors acknowledge that the guidance performance for the fuel-optimal problem is not as effective as for the time-optimal problem. This is attributed to the inherent sensitivity of the fixed-final-time fuel-optimal problem to initial state perturbations and the need to replicate both thrust direction and throttle.
  • Chattering Phenomenon: The thrust direction control can exhibit chattering, especially as the spacecraft approaches the target. A two-stage rendezvous strategy is suggested to mitigate this.
  • Future Work: The authors suggest focusing on enhancing the guidance performance for the fuel-optimal problem. Further research could explore more advanced techniques to address the sensitivity to perturbations and improve the replication of complex throttle profiles.

Summary updated

Abstract

Machine learning techniques have demonstrated their effectiveness in achieving autonomy and optimality for nonlinear and high-dimensional dynamical systems. However, traditional black-box machine learning methods often lack formal stability guarantees, which are critical for safety-sensitive aerospace applications. This paper proposes a comprehensive framework that combines control Lyapunov functions with supervised learning to provide certifiably stable, time- and fuel-optimal guidance for rendezvous maneuvers governed by Clohessy-Wiltshire dynamics. The framework is easily extensible to nonlinear control-affine systems. A novel neural candidate Lyapunov function is developed to ensure positive definiteness. Subsequently, a control policy is defined, in which the thrust direction vector minimizes the Lyapunov function's time derivative, and the thrust throttle is determined using minimal required throttle. This approach ensures that all loss terms related to the control Lyapunov function are either naturally satisfied or replaced by the derived control policy. To jointly supervise the Lyapunov function and the control policy, a simple loss function is introduced, leveraging optimal state-control pairs obtained by a polynomial maps based method. Consequently, the trained neural network not only certifies the Lyapunov function but also generates a near-optimal guidance policy, even for the bang-bang fuel-optimal problem. Extensive numerical simulations are presented to validate the proposed method.
BibTeX
@misc{wang2025learning,
  author = {Kun Wang and Roberto Armellin and Adam Evans and Harry Holt and Zheng Chen},
  title = {{Learning-Based Stable Optimal Guidance for Spacecraft Close-Proximity Operations}},
  year = {2025},
  eprint = {2501.01096},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2501.01096},
}
Figure