TL;DR
AeroRSI is a recursive self-improvement framework that allows aerial drones to learn from their own navigation failures without human intervention. By having a model "test-fly" potential corrections and only keeping those that lead to successful mission completion, the system iteratively reveals and fixes hidden failures, boosting success rates on the TravelUAV benchmark from 46.4% to 67.5%.
What Problem Does This Paper Address?
Vision-Language-Action (VLA) models allow drones to follow natural-language instructions (e.g., "Fly past the red building and land near the fountain"). However, these models frequently fail in "closed-loop" flight—meaning they make a small mistake, end up in a state they weren't trained for, and then crash or get stuck.
Correcting these errors is difficult for two reasons:
- Labeling Scarcity: Human pilots cannot easily provide labels for the infinite variety of "mistake states" a drone might reach.
- Hidden Failures: An episode ends at the first failure (like a collision). This hides subsequent failures further down the path. You can't fix a mistake at the end of a flight if the drone always crashes at the beginning.
Methods
The authors introduce AeroRSI, a framework that treats self-improvement as a recursive loop. Instead of relying on an external expert, the failing policy itself acts as the judge of its own "repairs."
The process consists of three main modules per round:
- Adaptive Failure Anchoring (AFA): The system identifies "anchor states" leading up to a failure. It places probes at different time scales—some near the crash to catch immediate causes, and some further back to catch poor strategic decisions.
- Continuation-Verified Repair (CVR): For every anchor state, the model generates candidate actions from its own samples or simple rule-based proposals (like "climb higher"). Crucially, it verifies these candidates by executing the action and letting the current policy attempt to finish the flight. If the drone reaches the goal, that candidate action is saved as a "verified repair."
- Retentive Repair Learning (RRL): The model is updated using the verified repairs. To prevent "catastrophic forgetting" or drifting away from good behaviors, it uses a retention loss that anchors the model to its own successful greedy outputs on states that didn't need repair.
Datasets
The researchers evaluated AeroRSI on TravelUAV, a high-fidelity benchmark for aerial navigation built on AirSim and Unreal Engine. It includes:
- 7,611 training routes across 11 diverse environments.
- Test sets categorized by "Seen" environments, "Unseen Objects," and "Unseen Maps" to test generalization.
Key Insights and Ideas
- The Policy as its Own Judge: The "aha moment" is that a failing policy is actually a great judge of repairs. If a candidate action allows the current (imperfect) policy to successfully complete the rest of the flight, that action is a highly reliable supervision signal.
- Recursion is Essential: As shown in Figure 1, fixing an early failure (A) reveals a "hidden" failure (B) further along the path. By repeating the process across multiple rounds, the model "peels the onion" of its own incompetence.
- Turning Weak Signals into Gold Labels: Simple rule-based proposals (like a potential-field avoidance move) might actually hurt performance if imitated blindly. However, once verified by a successful flight continuation, they become high-quality training data.
Key Results
- Significant Performance Boost: Starting from a supervised 2B-parameter model, AeroRSI raised the success rate from 46.4% to 67.5% in just three rounds.
- New State-of-the-Art: It outperformed the previous best method (AeroDPO) by 10.6 points using the exact same initialization and without any extra human labels.
- Collision Reduction: The framework effectively taught the drone to fly higher and more cautiously, reducing the collision rate by approximately 50%.
- Failure Shift: Over successive rounds, the primary cause of failure shifted from "Collisions" to "Timeouts," suggesting the model became much safer but occasionally too slow to reach the target.
Limitations and Future Work
- Simulation Cost: While AeroRSI eliminates human labeling, it is computationally expensive in simulation. Each round requires thousands of "test flights" to verify repairs.
- Sim-to-Real Gap: The method relies on the ability to "save and restore" states in a simulator to test different actions from the exact same moment. This cannot be done directly in the real world, though the resulting policy can be deployed on real hardware.
- Language Grounding: While navigation improves, the paper doesn't deeply explore if the model's actual understanding of complex linguistic nuances improves, or if it's primarily learning better spatial "common sense."
Summary updated