A quadrotor has 12 state variables and 4 motors, yet its whole motion is pinned down by four quantities: its 3D position and its heading. Engineers call this differential flatness. Given a smooth path for those four "flat outputs" and its time derivatives, simple algebra recovers the attitude, body rates, and motor commands. No simulation or integration is needed. Controllers built on this map are precise when the model is right. They fail when unmodeled forces appear, such as rotor drag, a shifted center of gravity, or motor lag. Learned controllers trained in randomized simulation show the opposite profile: they recover well from disturbances, but they must rediscover basic flight physics from scratch, and the outputs they are allowed to produce largely decide whether they survive on real hardware.
We propose to keep the flatness map exact for everything that can be measured, such as mass, inertia, and geometry, and to let a small policy learn only the part the model gets wrong. The policy is trained by reinforcement learning (RL) in simulation under domain randomization, meaning simulator physics are varied at random across episodes so the real robot falls inside the training range. It outputs a 6-number estimate of the unmodeled force and moment, and that estimate enters the flatness inversion at the physically correct place. The bet is that this placement matters for three reasons. Commands stay consistent with the known dynamics. Randomization can stay restricted to quantities that are genuinely unknown. And because the outputs are normalized as accelerations rather than motor commands, a single policy can transfer across airframes with no real-world training data at all.
Core Claim
The flatness inversion still holds when known external force and moment terms are added to it. With $\mathbf{a}$ the commanded flat-output acceleration, $g\mathbf{z}_W$ gravity, $m$ and $\mathbf{J}$ the identified mass and inertia, $\boldsymbol{\omega}, \dot{\boldsymbol{\omega}}$ the body rates and angular accelerations obtained from the reference jerk and snap, and $\hat{\mathbf{d}}_f, \hat{\mathbf{d}}_\tau$ the policy's estimates of unmodeled acceleration and angular acceleration:
$$ f_T\,\mathbf{z}_B = m\,(\mathbf{a} + g\mathbf{z}_W - \hat{\mathbf{d}}_f), \qquad \boldsymbol{\tau} = \mathbf{J}\,(\dot{\boldsymbol{\omega}} - \hat{\mathbf{d}}_\tau) + \boldsymbol{\omega}\times\mathbf{J}\boldsymbol{\omega}. $$
Here $f_T$ is collective thrust, $\mathbf{z}_B$ the body vertical axis, and $\boldsymbol{\tau}$ the body torque. The claim has two parts. First, a policy whose only job is to output $(\hat{\mathbf{d}}_f, \hat{\mathbf{d}}_\tau)$, trained against a tracking reward under randomization of only the unmeasurable parameters, keeps the near-exact nominal tracking of flatness control and gains the disturbance recovery of RL. Second, because its outputs are in mass- and inertia-normalized units, one trained policy should transfer zero-shot across quadrotors of different size once their mass and inertia are identified.
The moment channel is not optional. A constant center-of-gravity offset needs a sustained torque bias. Correcting the flat-output reference alone cannot produce one, because the nominal map assigns zero torque to steady hover. This is why we correct the inversion's force and moment terms instead of modifying the reference path.
Relation to Prior Work
Three lines of work motivate the gap. Leveling the Playing Field compares a flatness-based geometric controller against a PPO policy in IsaacLab under matched objective, data, and look-ahead. It finds RL far better on reactive tasks and the classical controller degraded under 20% mass and inertia variation, but it treats the two as rivals and never deploys either on hardware. An Adaptive Control Strategy for Neural Network based Optimal Quadcopter Controllers shows on a real Bebop that a flatness-based controller saturates its motors and crashes under a 0.06 Nm center-of-gravity moment, while a neural controller fed an online moment estimate survives. That network outputs raw motor speeds, trains without randomization, and estimates only the moment, using a hand-derived angular-acceleration comparison. Exploiting Differential Flatness for Robust Learning-Based Tracking Control Using Gaussian Processes shows that differentiating noisy positions up to snap destroys continuous flatness control at speed, and it names learning the flatness maps from data as open work. We do not learn the map itself. We learn only its correction terms, which avoids differentiating measurements because the derivatives come from the reference path.
Two papers constrain the design. What Matters in Learning A Zero-Shot Sim-to-Real RL Policy for Quadrotor Control? finds that randomizing measurable mass and inertia hurts real transfer, and that only unmeasurable thrust coefficients should be randomized. Our structure makes this split natural: measured parameters live inside the inversion, and unmeasured ones are absorbed by the learned terms. The Power of Input finds that minimal observations (position error, body rates, previous action) transfer better than the full state. We adopt its minimal set plus a short history of commanded-versus-measured body rates, and we avoid feeding derivatives up to snap.
Two papers supply the method. Residual control, as in Simulation-based reinforcement learning for real-world autonomous driving, adds an RL correction on top of a classical command. We change where that correction enters. On the Role of the Action Space in Robot Manipulation Learning and Sim-to-Real Transfer shows on a Franka arm that the action interface alone can swing real success from 6% to 96%. That is the empirical reason to expect the placement of the residual, and not just its presence, to matter.
Research Program
The system has three parts. The first is a nominal planner that produces minimum-snap paths in flat-output space, following Search-based Motion Planning for Aggressive Flight in SE(3). The second is the disturbance-augmented inversion above, which maps the path and the learned terms to collective thrust and body rates. When the requested thrust exceeds motor limits, an algebraic feasibility check keeps thrust direction and sacrifices yaw first. The third is a small policy that outputs $(\hat{\mathbf{d}}_f, \hat{\mathbf{d}}_\tau)$ through a first-order low-pass filter. The filter gives smoothness by construction, so no action-rate penalty is needed.
Training uses PPO in OmniDrones. Mass and inertia are fixed at identified values. Randomization covers thrust coefficient, linear rotor drag, center-of-gravity offset, motor time constant, and latency of 0–10 ms. The reward is tracking error plus a saturation penalty. To test cross-airframe transfer, training samples airframes across a range of sizes. The policy never sees absolute mass, because its outputs are normalized.
The analysis has one central question beyond performance: are the learned terms physically meaningful? In simulation, we will regress $\hat{\mathbf{d}}_f$ and $\hat{\mathbf{d}}_\tau$ against the true injected disturbance. On hardware, we will compare $\hat{\mathbf{d}}_\tau$ against the known moment produced by an attached offset mass.
Baselines
The naive reference point is the geometric flatness controller with careful identification of mass, inertia, and thrust curves and no adaptation. It is the controller the proposal modifies, so beating it measures the value of the learned terms. Leveling the Playing Field reports its Lissajous position RMSE at 0.158 m, against 0.119 m for PPO. The file describes the classical controller as having near-zero steady-state error, but on this reported RMSE the learned policy was lower. We will report steady-state error and full-path RMSE separately to resolve this. The same paper reports ball-catching success of 30% for this controller and 65% for RL.
The strongest classical competitor is the same geometric controller augmented with a hand-designed disturbance observer. For moments, it compares measured angular acceleration to the nominal prediction, as in the Bebop adaptive controller. For forces, it uses the analogous translational comparison. It shares our exact structure and differs only in how $(\hat{\mathbf{d}}_f, \hat{\mathbf{d}}_\tau)$ is produced, so it is the decisive ablation of the learned ingredient. Its filter bandwidths get the same Bayesian-optimization budget, against the same reward, that Leveling the Playing Field gives its controller gains.
The strongest learned competitor is a SimpleFlight-style end-to-end policy outputting collective thrust and body rates, trained with that paper's recommendations including selective randomization. SimpleFlight reports more than a 50% tracking-error reduction over prior RL baselines and is the only method to complete the fast trajectories. Absolute error values are not given, so for SimpleFlight's raw tracking error we found no reported result on Lacuna; we will reproduce it ourselves. The best reported real-world tracking figure in the file is 3.9 cm on a 3D figure-eight (The Power of Input, versus 4.9 cm for PID), and we use it as a reference scale.
A second ablation is residual RL in input space: the policy adds a correction directly to the geometric controller's thrust and body-rate output, with identical observations, randomization, and environment steps. It isolates whether the inversion-level placement matters. A third ablation trains our structure with full randomization, including mass and inertia, to test whether the selective-randomization finding survives once measured parameters sit inside a model.
All methods share simulator, trajectory distribution, look-ahead horizon, and seeds. Learned methods get equal environment steps and an equal hyperparameter-search budget. The proposal counts as winning if two things hold. It must match the geometric controller's steady-state error on nominal paths while matching or exceeding the end-to-end policy on reactive and disturbed tasks. And it must beat the disturbance-observer baseline on at least the time-varying disturbance conditions.
Why not just…?
Why not just add a disturbance observer to the geometric controller? This is the real threat. For constant offsets such as a fixed center-of-gravity shift, a well-tuned observer may match a learned estimator, because the Bebop result shows a hand-derived moment estimate already suffices to prevent the crash. The learned estimator's possible advantages are narrower. It can use the reference look-ahead to anticipate velocity-dependent drag before it shows up in measurements, and it is tuned against the task reward rather than an estimation loss. We will separate these effects with conditions where an observer must lag: step changes in drag, fast pentagram and zigzag paths, and the ball-catching task. If the observer ties everywhere, the honest conclusion is that the structure, not the learning, carries the result, and we will report it that way.
Why not just learn a residual on the controller's output? The two are equally expressive, since the inversion is invertible away from zero thrust. Any difference must come from conditioning: inversion-level outputs are in physical units, are low-pass filtered, and are normalized by mass and inertia. The action-space study on the Franka arm shows that interface choice alone can move real success between 6% and 96%, so a gap is plausible but not guaranteed here. The input-space ablation, run with matched steps, will show differences in learning speed, command smoothness on hardware, and especially cross-airframe transfer, where output-level residuals in absolute thrust units should not carry over.
Why not just use SimpleFlight's end-to-end recipe? It already transfers zero-shot to a Crazyflie and a larger quadrotor. The question is whether a policy that must also rediscover the known part of the dynamics wastes capacity and drifts on steady tracking. The steady-state versus transient split in Leveling the Playing Field suggests it might. The comparison on nominal versus disturbed paths, at equal environment steps, will show whether that cost appears.
Proposed Experiments
In simulation, we evaluate on OmniDrones and the IsaacLab tasks from Leveling the Playing Field: hover, Lissajous tracking, and ball catching. We add SimpleFlight's fast pentagram and zigzag paths and an injected sweep of drag and center-of-gravity offset beyond the training range. Metrics are steady-state error and full-path position RMSE, catch success, time spent in motor saturation, and learning curves against environment steps.
On hardware, a Crazyflie under motion capture flies the figure-eight from The Power of Input and the fast SimpleFlight paths. The tests are run with and without an attached offset mass sized to give a moment proportionate to the 0.06 Nm Bebop test, scaled to the Crazyflie's size. A second, larger frame uses the identical policy, with only mass and inertia re-identified.
The decisive checks are four. Does our method stay below the flatness controller's saturation-driven failure under the offset mass? Does it beat the disturbance observer on time-varying disturbances? Does the single policy keep its tracking advantage on the second airframe, where the input-space residual degrades? And do the learned moment estimates on hardware agree with the known offset to within a measurable tolerance? If the last check fails while performance holds, the policy is doing something other than estimating disturbances, and the interpretability part of the claim would be withdrawn.
References
- Leveling the Playing Field: Carefully Comparing Classical and Learned Controllers for Quadrotor Trajectory Tracking
- An Adaptive Control Strategy for Neural Network based Optimal Quadcopter Controllers
- Exploiting Differential Flatness for Robust Learning-Based Tracking Control Using Gaussian Processes
- What Matters in Learning a Zero-Shot Sim-to-Real RL Policy for Quadrotor Control? A Comprehensive Study
- The Power of Input: Benchmarking Zero-Shot Sim-to-Real Transfer of Reinforcement Learning Control Policies for Quadrotor Control
- Simulation-based reinforcement learning for real-world autonomous driving
- The Role of Action Abstractions in Robot Manipulation Learning and Sim-to-Real Transfer
- Search-Based Motion Planning for Aggressive Flight in SE(3)
- OmniDrones: An Efficient and Flexible Platform for Reinforcement Learning in Drone Control