Distances That Remember: Hysteresis-Aware Quasimetrics for Goal-Reaching Flight

Many learning agents decide what is "close" by measuring straight-line distance between where they are and where they want to be. For an aircraft this is often wrong, and wrong in two separate ways. The first is that wind makes travel one-directional. Reaching a target speed or heading against a crosswind costs more time and energy than the reverse maneuver. The second, which is less obvious, is that the wing's air remembers the recent past. A wing pitching up through a high angle of attack (the angle between the wing and the oncoming air) keeps its airflow attached and still produces lift. The same wing passing through the same angle on the way down has already stalled and produces much less. Two aircraft states that look identical on the instruments can therefore be very different distances from a goal, depending on what the air over the wing is doing.

This note proposes to learn goal-reaching distances for fixed-wing flight that account for both effects. The distance is asymmetric, so the cost from A to B need not equal the cost from B to A. It is also conditioned on a short history of flight, so it can tell attached flow from stalled flow. The first property already has good tools: quasimetric reinforcement learning. The open question is whether the second property matters for planning, and if so, what representation it needs. We will answer this with a computable ground truth on a flight simulator augmented with wind and a hysteretic stall model, rather than assuming the answer.

Core Claim

Quasimetric methods learn a distance function that obeys the triangle inequality but may be asymmetric. In deterministic settings, such a function provably equals the optimal goal-reaching cost (QRL). These methods measure distance between memoryless states. In dynamic stall that assumption breaks: the hidden flow state (attached, carrying a leading-edge vortex, or separated) changes the cost-to-go, and the flight instruments do not report it.

We claim three things. First, in wind with stall-prone maneuvers, the true cost-to-go has a large asymmetric component and a large flow-state-dependent component. Second, symmetric learned distances and plain Euclidean distance mis-rank intermediate waypoints in ways that measurably hurt long-horizon goal reaching. Third, a quasimetric whose source embedding reads a short state history and whose goal embedding reads only the instrument-level target recovers most of the lost performance. The design choice that defines the method is this split between the two roles: where the aircraft starts includes the hidden flow state, while the goal leaves the flow state free.

The closest existing work is QuasiNav. It learns quasimetric embeddings for ground robots on slopes, where gravity makes uphill travel costlier, and reports 94% success against a 90.5% baseline. Its asymmetry is static and depends only on location. Ours is driven by flow and depends on history, and the question we ask is whether memory belongs inside the distance at all.

Why the pieces exist but have not been joined

The aerodynamic evidence motivates the problem. Surface-pressure measurements on yawed wind-turbine rotors show strong azimuthal asymmetry in blade loading (Kamada et al.). The same asymmetry produces restoring yaw moments (Maeda et al.). These are clean cases of oblique flow making forces direction-dependent. For memory, eddy-resolving simulations of pitching airfoils (IDDES dynamic stall study) and a convolutional reduced-order model of NACA 0012 dynamic stall (Baldan & Guardone) both show wide lift and moment hysteresis loops. The reduced-order model needed dedicated periodic temporal layers to reproduce those loops. NeuroBEM fuses blade-element physics with a temporal network over a 50 ms history, cutting quadrotor force errors by about half. That result tells us that in agile flight, memoryless models of aerodynamics fail. None of these papers measure or learn reachability.

The reachability literature supplies the method and its constraints. QRL and Interval Quasimetric Embeddings (IQE) give a parameter-efficient asymmetric distance head with approximation guarantees. TMD distills quasimetric distances from suboptimal offline data. We reuse IQE as the distance head and TMD-style offline training.

Symmetric alternatives define what we must beat. Bisimulation pseudometrics (PLOFF), actionable representations (ARC), and commute-time Laplacian embeddings (RA-LapRep) all place states in a symmetric Euclidean latent space. RA-LapRep explicitly requires reversible transitions.

Eikonal value regularization (Eik-HIQL) acts as a constraint. It lifts Antmaze-Giant success from 7% to 79%, but it assumes a smooth value field. A stall boundary is exactly where the value field jumps, so we treat Eik-HIQL as a strong baseline that we predict will degrade near stall, not as a component of our method.

RL-RRT shows that replacing Euclidean distance with a learned time-to-reach estimator in a sampling-based planner helps ground robots. This supports our downstream planning test.

Research Program

The environment. We extend VVC-Gym, a 6-degree-of-freedom fixed-wing simulator running at about 3600 frames per second. Its tasks command airspeed, climb angle and heading. Its authors state that it omits wind and asymmetric aerodynamics, and we add both. Wind enters as a steady field plus a parametric gust process, at several strengths.

Stall enters as a single hidden separation variable that lags the angle of attack. Its time constants differ on the upstroke and the downstroke, which produces a hysteresis loop. We calibrate the loop's shape and its dependence on reduced frequency against the NACA 0012 hysteresis loops produced by the Baldan & Guardone reduced-order model, so the loops have physically plausible proportions and are not hand-tuned. We will release the extension alongside the existing fly-craft-examples code.

A ground-truth distance. A low-dimensional longitudinal-lateral slice of the dynamics includes airspeed, climb angle, heading, angle of attack, the separation variable, and the wind parameter. On this slice we compute the true minimum time-to-goal by dynamic programming on a fine grid. This gives us three diagnostics. The first is the size of the asymmetry, $|d(A,B)-d(B,A)|$, where $d(A,B)$ is the optimal time from state $A$ to state $B$. The second is the size of the hysteresis effect: how much $d$ changes when only the hidden separation variable changes. The third is the accuracy of every learned distance against this ground truth.

This step is the honest gate for the project. If hysteresis changes the cost-to-go by only a few percent, the memory component is unnecessary. In that case we will report it and reduce the contribution to wind-driven asymmetry alone.

The method. A source encoder $f(s_{t-k:t})$ reads the last $k$ instrument-level states; $k$ is swept from 0 to roughly one pitch period. A goal encoder $\psi(g)$ reads only the commanded target. The learned distance is $d_\theta = \mathrm{IQE}(f(s_{t-k:t}), \psi(g))$, where $\mathrm{IQE}$ is the interval quasimetric head.

Using a waypoint as an intermediate stop requires it to play both roles. A waypoint $w$ therefore enters the triangle inequality as $\psi(w)$ when it is a target, and as $f$ applied to a short rollout history ending at $w$ when it is a starting point. This keeps the triangle inequality well defined while letting the hidden flow state at the waypoint be decided by how the aircraft arrives there.

Training follows TMD's offline recipe on the VVC-Gym demonstration sets, re-collected under wind and stall, plus suboptimal rollouts. The policy is extracted from the distance as in QRL.

Baselines

The naive reference is VVC-Gym's own setup: Euclidean distance in velocity-vector space as the reward, trained with PPO. Its authors report 0% success on the Hard goal space without demonstrations. We rerun it on our wind-and-stall variant at the same sample budget.

The demonstration baseline is goal-conditioned behavioral cloning (GCBC) on the same re-collected demonstrations. It is the strongest reported competitor on this exact platform: VVC-Gym reports over 70% success on Hard goals in calm air when demonstrations are used. We beat it only if we exceed GCBC's success on wind-and-stall Hard goals using the same demonstration data plus suboptimal data. We also expect larger margins on goals outside the demonstration envelope, where cloning cannot stitch new routes.

The strongest existing distance-learning method is TMD with a memoryless IQE or MRN head. No reported result on Lacuna was found for TMD, QRL or any quasimetric method on fixed-wing flight. The nearest reported numbers come from other domains: QRL's more than 37% gain over the best baseline on offline maze2d, and Eik-HIQL's 79% versus 7% on Antmaze-Giant. We run Eik-HIQL and HIQL with the authors' settings as the strongest value-regularized competitors.

All methods share the same datasets and network width. Each gets the same budget of hyperparameter trials, a fixed number per method chosen on a held-out wind seed, with five training seeds each. The ablation that isolates our key ingredient keeps everything fixed and sets the history length $k=0$. A second ablation keeps the history but replaces IQE with a symmetric Euclidean head, in the style of ARC. Together these two separate the value of memory from the value of asymmetry.

Why not just…?

Why not just add a wind-heading penalty to Euclidean distance? A term proportional to the alignment between the commanded heading change and the wind is cheap and might capture first-order asymmetry. It cannot see stall at all, and it does not enforce the triangle inequality, which QRL identifies as what prevents overestimation when routes are stitched through waypoints. We include this baseline with its coefficient tuned on the ground-truth grid. If it matches the learned quasimetric in steady wind, we will say so and confine our claim to the stall regime.

Why not just stack frames into an off-the-shelf quasimetric learner? This is the real threat, and it is close to what we propose. The differences are the goal encoder that leaves the flow state free and the explicit sweep over history length. Naively stacking frames on both the source and the goal would force goals to specify a flow state the commander never chooses. We test this directly with a variant that uses symmetric frame stacking on both roles. If it performs as well as our split encoders, the contribution reduces to the empirical finding that memory matters for flight distances, plus the benchmark. That would still be useful, but we will state it as the smaller result it is.

Why not just fly a model-predictive controller on a good aerodynamic model? NeuroBEM shows that pure blade-element physics leaves tracking errors near 0.8 m that a learned residual cuts below 0.3 m, and each physics evaluation adds about 100 µs. A short-horizon controller also cannot rank distant waypoints across a stall excursion. We include a model-predictive controller with access to the true simulator dynamics as an oracle upper bound on local tracking. It serves only as an oracle, because no real aircraft has that model.

Why not just use behavioral cloning with more demonstrations? Cloning copies the routes it was shown. Our held-out evaluation uses start–goal pairs that require stitching segments across wind directions never paired in the demonstrations. On those pairs TMD's argument predicts that cloning fails and distance distillation does not. The experiment will show whether that holds in flight.

Proposed Experiments

The first experiment is the diagnostic on the ground-truth grid. We report the fraction of cost-to-go variance explained by asymmetry and by the hidden separation variable as wind strength and maneuver aggressiveness vary. We also report each learned distance's rank correlation with ground truth, measured separately in attached-flow and near-stall regions. We predict Eik-HIQL's error concentrates at the stall boundary and that $k=0$ quasimetrics show a gap between upstroke and downstroke states that history removes.

The second experiment is goal reaching on the extended VVC-Gym Normal and Hard tiers, in calm air, steady crosswind and gusts. It reports success rate, time-to-goal relative to the dynamic-programming optimum where that optimum is available, and the rate of entry into deep stall.

The third experiment plugs each learned distance into a sampling-based planner, following RL-RRT, to select tree nodes for multi-waypoint flight. It reports success within a fixed planning budget and executed path time.

The decisive check is the combination of two results: the $k=0$ versus $k>0$ ablation must open a gap in the stall regime and close it in calm attached flow, and the ground-truth diagnostic must confirm that hidden flow state truly changes reachability. If either fails, the memory claim is falsified and the contribution reduces to the wind-driven asymmetry.