Coordinated Reentry Guidance with A* and Deep Reinforcement Learning for Hypersonic Morphing Vehicles Under Multiple No-Fly Zones

Cunyu Bao, Xingchen Li, Weile Xu, Guojian Tang, Wen YaoAerospace 2025 · DOI

TL;DR

This paper presents a novel hierarchical guidance system for hypersonic morphing vehicles (HMVs) during reentry. It combines A* search for no-fly zone avoidance, Deep Deterministic Policy Gradient (DDPG) for adaptive morphing control, and a Quasi-Equilibrium Glide Condition (QEGC) guidance law. This integrated approach significantly improves trajectory planning and control accuracy, ensuring safe and efficient reentry even under challenging conditions.

What Problem Does This Paper Address?

Hypersonic morphing vehicles (HMVs) are advanced aircraft capable of changing their shape mid-flight to optimize aerodynamic performance. However, during reentry, they face complex challenges: navigating through multiple "no-fly zones" (areas to be avoided), adhering to strict path constraints (like heat rate and G-force limits), and dealing with highly nonlinear dynamics that are further complicated by the morphing itself. Traditional guidance systems struggle with these combined issues, especially when no-fly zones can change dynamically. This paper aims to develop a robust and intelligent guidance system that can handle these complexities by integrating global trajectory planning with real-time, adaptive control.

Methods

The proposed solution is a three-layered hierarchical framework:

  1. Planning Layer:
    • A* Search Algorithm: This algorithm is used to plan an energy-optimal trajectory that avoids multiple no-fly zones. It systematically searches for waypoints, prioritizing those that minimize energy expenditure while ensuring safe passage. This is an improvement over greedy methods, showing a 6.2% reduction in the evaluation function.
  2. Deep Deterministic Policy Gradient (DDPG): This deep reinforcement learning algorithm is employed to learn an intelligent morphing policy. The DDPG agent learns to adjust the vehicle's sweep angle in real-time to optimize aerodynamic performance, minimize velocity loss, and improve tracking accuracy, especially in response to changing flight conditions.
  3. Guidance Layer:
    • Quasi-Equilibrium Glide Condition (QEGC) Guidance Law: This law is designed to ensure precise longitudinal and lateral tracking. It uses a continuous switching sliding mode control with a hyperbolic tangent function to achieve smooth and robust control, transforming path constraints into direct guidance input constraints.
  4. Output Layer:
    • The system integrates the outputs from the planning and guidance layers to determine the final control inputs: angle of attack, bank angle, and sweep angle, while respecting input constraints.

Datasets

Not discussed in the paper. The simulations appear to use a physics-based model of the HMV and its environment, with parameters adapted from existing hypersonic vehicle references.

Key Insights and Ideas

  • Hierarchical Integration is Key: The paper demonstrates that effectively combining global path planning (A*) with adaptive, real-time control (DDPG) is crucial for complex reentry scenarios.
  • DDPG for Adaptive Morphing: Using DDPG to learn a morphing policy allows the HMV to dynamically adjust its shape (sweep angle) to optimize for different flight objectives (e.g., minimizing velocity loss, improving tracking) in real-time, a capability that fixed morphing strategies lack.
  • A* for Efficient No-Fly Zone Avoidance: The A* algorithm provides a systematic and energy-optimal way to navigate complex no-fly zone configurations, outperforming simpler search methods.
  • Robustness through Control Design: The QEGC guidance law with continuous sliding mode control ensures stable and accurate tracking, even under disturbances.
  • Computational Efficiency: The trained DDPG policy is computationally light, making it feasible for onboard deployment on modern embedded systems.

Key Results

  • Trajectory Planning: The A* algorithm reduced the evaluation function value by 6.2% compared to greedy methods in no-fly zone avoidance.
  • Terminal Accuracy: The proposed method achieved terminal errors of 0.09 km in position and 0.01 m/s in velocity.
  • Generalization: The system demonstrated good generalization across different target positions, with terminal position errors below 0.24 km and velocity errors below 5.90 m/s.
  • Robustness: Monte Carlo simulations showed resilience to deviations in initial states, atmospheric density, and aerodynamic coefficients, with terminal range errors of 0.263 ± 0.184 km and velocity errors of -12.7 ± 42.93 m/s.
  • Computational Performance: The A* planning completes in under 0.1 ms, and the DDPG policy achieves a 0.1 ms inference time, allowing for real-time operation within a 50 ms guidance cycle.

Limitations and Future Work

  • Limitations: The paper acknowledges that the current work focuses on the reentry phase. The simulation parameters were adapted from existing references due to the lack of publicly available HMV models.
  • Future Work: The authors suggest extending the framework to ascent and dive phases. They also aim to further optimize the computational efficiency of the policy network for onboard deployment on flight-grade embedded processors.

Summary updated

Abstract

Hypersonic morphing vehicles (HMVs), renowned for their adaptive structural reconfiguration and cross-domain maneuverability, confront formidable reentry guidance challenges under multiple no-fly zones, stringent path constraints, and nonlinear dynamics exacerbated by morphing-induced aerodynamic uncertainties. To address these issues, this study proposes a hierarchical framework integrating an A-based energy-optimal waypoint planner, a deep deterministic policy gradient (DDPG)-driven morphing policy network, and a quasi-equilibrium glide condition (QEGC) guidance law with continuous sliding mode control. The A* algorithm generates heuristic trajectories circumventing no-fly zones, reducing the evaluation function by 6.2% compared to greedy methods, while DDPG optimizes sweep angles to minimize velocity loss and terminal errors (0.09 km position, 0.01 m/s velocity). The QEGC law ensures robust longitudinal-lateral tracking via smooth hyperbolic tangent switching. Simulations demonstrate generalization across diverse targets (terminal errors < 0.24 km) and robustness under Monte Carlo deviations (0.263 ± 0.184 km range, −12.7 ± 42.93 m/s velocity). This work bridges global trajectory planning with real-time morphing adaptation, advancing intelligent HMV control. Future research will extend this framework to ascent/dive phases and optimize its computational efficiency for onboard deployment.
BibTeX
@inproceedings{bao2025coordinated,
  author = {Cunyu Bao and Xingchen Li and Weile Xu and Guojian Tang and Wen Yao},
  title = {{Coordinated Reentry Guidance with A* and Deep Reinforcement Learning for Hypersonic Morphing Vehicles Under Multiple No-Fly Zones}},
  booktitle = {Aerospace},
  year = {2025},
  doi = {10.3390/aerospace12070591},
}
Figure