Abstract
Traditional approaches to autonomous aerobatic flight rely on a decoupled architecture of trajectory optimization and path tracking, which often fails under the high-dynamic constraints and environmental uncertainties of extreme maneuvers. This paper presents an end-to-end reinforcement learning (RL) framework that maps sensor data directly to low-level motor commands. By utilizing a biased reset-based curriculum and a body-rate action space, the system achieves unprecedented reactivity. Experimental results demonstrate the first autonomous quadrotor capable of navigating moving gates while maintaining inverted flight, outperforming state-of-the-art optimization methods in both speed and tracking precision.
Introduction
Aerobatic flight represents the pinnacle of agile robotics, requiring a vehicle to operate at the very edge of its physical envelope. While human pilots can perform complex maneuvers like barrel rolls and power loops with fluid reactivity, autonomous systems have historically struggled with the latency and modeling errors inherent in multi-stage planning pipelines. This research introduces a model-free reinforcement learning approach that replaces traditional planning with a single, high-frequency neural network policy, enabling "reactive" aerobatics that can adapt to dynamic environments in real-time.
The Limitations of Traditional Planning
Conventional aerobatic control typically treats flight as a two-stage problem: first, optimizing a trajectory based on idealized physics, and second, employing a controller to track that path. This paradigm suffers from three critical vulnerabilities:
- Tracking Divergence: Real-world factors such as aerodynamic turbulence, battery voltage sag, and motor noise are difficult to model, causing the drone to drift from the pre-computed path during high-G maneuvers.
- Computational Latency: Re-calculating a trajectory in response to a moving target (e.g., a moving gate) can take seconds, which is orders of magnitude too slow for the millisecond-level adjustments required in aerobatics.
- Optimization Local Minima: Trajectory optimizers are sensitive to initial conditions and often converge on sub-optimal solutions that do not exploit the full agility of the airframe.
Methodology
Reinforcement Learning Framework
The core of the system is a policy trained via Proximal Policy Optimization (PPO). Unlike previous methods that output high-level position setpoints, this policy outputs collective thrust and body rates. This specific action space provides a "Goldilocks" balance: it is high-level enough to be robust against sim-to-real noise, yet low-level enough to allow the agent to exploit the quadrotor's maximum angular acceleration.
Automated Curriculum Learning
Learning aerobatics from scratch is a "sparse reward" problem; a drone is statistically unlikely to complete a barrel roll through random exploration. To solve this, the authors implemented Biased Reset-Based Training. Using the principle of differential flatness, the system analytically identifies states along a desired maneuver and starts the agent near the goal. As the agent's performance improves, the starting points are moved progressively further back, effectively "guiding" the policy through the complex physics of the maneuver.
Sim-to-Real Transfer
To bridge the gap between simulation (using the Flightmare engine) and reality, the researchers employed extensive domain randomization. They perturbed air drag coefficients, mass distributions, and control latencies during training. Furthermore, system delay was mitigated by averaging network outputs over a sliding window of past time steps, ensuring stable control on physical hardware.
Experimental Results
Performance Benchmarks
The RL-based policy was evaluated against a state-of-the-art optimization baseline across three benchmark tracks: the Power Loop, Barrel Roll, and Split-S. The RL agent completed these tracks 27% faster on average (12.3s vs 16.9s). More importantly, the RL approach maintained significantly lower position and attitude errors, demonstrating that an end-to-end policy can find more efficient and robust control laws than manual optimization.
Reactivity and Robustness
The system's reactivity was tested by moving waypoints during flight. The drone maintained a 78% success rate even when targets moved at 3.4 m/s, a speed significantly higher than any dynamic element encountered during the training phase. In real-world hardware tests using a Jetson Orin NX-equipped quadrotor, the system successfully navigated a moving gate while inverted—a first in the field of autonomous flight. The maximum tracking error recorded was 0.35m, compared to 0.95m for the baseline.
Discussion and Future Work
While this framework achieves a response time of just 0.3 ms, several challenges remain. The current implementation relies on an external Motion Capture System (MCS) for state estimation; transitioning to purely onboard vision-based sensing is a necessary next step for field deployment. Additionally, while the execution of maneuvers is autonomous, the "intention" (the sequence of waypoints) is still human-defined. Future research may explore the use of Generative AI to autonomously design and chain together novel aerobatic maneuvers, pushing the boundaries of robotic creativity and agility.
Summary updated