Fault-Indexed Dynamics Priors for Safe, Fast Recovery After Actuator Failure

When a motor seizes or a propeller cracks, a learning-based controller faces two problems at once. Its model of how the robot moves is suddenly wrong. Any trial-and-error it does to fix that model could crash the machine. Classical fault-tolerant control handles the first moment well: fast estimators detect an 80% propeller loss in 0.15 seconds. But these controllers keep a fixed hand-written model and cannot learn what that model misses. Learning-based safe control, in turn, needs many samples and assumes the world it learned is still the world it is in.

We propose to join the two at a specific point. Train a learned dynamics model in simulation across many simulated faults, indexed by a short physical "fault vector" (how much effectiveness each actuator retains). At deployment, let a classical fault estimator fill in that vector the moment a fault occurs. The controller then plans with an already-adapted model instead of a broken one, and only a small residual error has to be learned online, inside a safety margin set by how uncertain the estimator and the residual are. The bet is that this turns post-fault learning from a large, dangerous correction into a small, safe one.

Core Claim

The claim has two parts, and both are falsifiable. First, the size of the online correction a learning controller must make after a fault depends mainly on where it starts. A model re-indexed by a fault estimator starts close to the post-fault dynamics. A model learned around the healthy plant starts far away. So the indexed model should need several times fewer post-fault samples to regain tracking accuracy than a correction learned from the healthy model.

Second, the safety margin should reflect this. If the margin is tightened by the fault estimate's error bound plus the residual model's predictive variance, the controller can let the healthy actuators run all the way to saturation while still keeping the robot's state inside its limits. That addresses the failure mode in which conservative safe learners refuse to use the authority they have left.

Concretely, the predicted next state is

$$s_{t+1} = f_0(s_t, a_t) + g_\phi(s_t, a_t; \hat\theta_t) + h_t(s_t, a_t),$$

where $f_0$ is the nominal physics model, $g_\phi$ is a neural residual with parameters $\phi$ trained offline across faults, $\hat\theta_t$ is the estimated per-actuator effectiveness vector at time $t$, and $h_t$ is a Gaussian process (a nonparametric regressor that also reports its own uncertainty) fitted online to whatever error remains. A model predictive controller plans actions over a short horizon with this model. It enforces the state constraint $c(s) \le 0$ in tightened form:

$$c(\hat s_{t+k}) + \beta\big(\sigma_\theta(t) L_\theta + \sigma_h(\hat s_{t+k}, a_{t+k})\big) \le 0.$$

Here $\hat s_{t+k}$ is the predicted state $k$ steps ahead, $\sigma_\theta$ is the estimator's error bound, $L_\theta$ is a sensitivity constant measuring how much the prediction moves per unit error in $\theta$, $\sigma_h$ is the Gaussian process standard deviation, and $\beta$ is a confidence multiplier. Actuator limits are hard input bounds, but no extra margin is placed on actuator usage. The margin lives on the state, which is what actually crashes.

Relation to Prior Work

Two lines of work supply the method, and each is missing the other's ingredient.

Fast fault estimation without learning comes first. From Propeller Damage Estimation and Adaptation to Fault Tolerant Control combines $\mathcal{L}_1$ adaptive control (an adaptive scheme that estimates disturbances quickly while filtering out high-frequency chatter) with an optimization step that estimates propeller damage within 4% error. Nonlinear MPC for Quadrotor Fault-Tolerant Control recovers from total single-rotor loss using a full nonlinear model. Both keep the model fixed apart from the fault parameters. The NMPC authors themselves needed an extra inversion layer to absorb unmodeled aerodynamics of damaged flight. We reuse such an estimator as the source of $\hat\theta$, and we add learning for what it cannot represent.

Sample-efficient learned correction without fault awareness comes second. safe-control-gym shows that a Gaussian process wrapped around an imperfect prior model (GP-MPC) needs roughly a tenth of the data of PPO or SAC. That benchmark evaluates parameter errors and wind, not actuator failure. Its correction always starts from the healthy nominal model. Our proposal differs exactly there: the starting model is moved by the fault estimate before any online data arrive.

Several papers motivate the gap. SafeDreamer gets near-zero safety cost by planning in a learned world model, but that model is stationary and has no slot for a fault. After a failure it keeps predicting healthy behavior, so its safety becomes imagined rather than real. Provably Efficient Model-Free Algorithms for Non-stationary CMDPs tightens constraints to obtain zero violations under change, but its guarantees assume a bounded variation budget across episodes. They do not cover an abrupt jump inside an episode. Our tightening borrows the idea of a margin, but it indexes the margin by the estimator's error at the current moment.

Breaking Safety Paradox with Feasible Dual Policy Iteration constrains the design. It shows that safer policies starve the learner of data near the boundary. We avoid that dependence by letting the physics prior and the estimator, rather than violations, define where the boundary is.

Domain-randomized fault-tolerant policies are the competing approach rather than a component. Saving the Limping and Adaptive Curriculum Dynamics Randomization train one policy across sampled failures and need no online samples at all. However, they enforce no constraints during training, and they cover only failure modes seen in simulation.

Research Program

The work has three parts.

The first part builds fault-injection variants of existing benchmarks. In safe-control-gym we add per-actuator effectiveness multipliers, stuck-at faults, and thrust-dependent degradation to the 2D quadrotor. We also add an over-actuated planar variant with four tilted thrusters, so that losing one thruster leaves the robot controllable through reallocation. This matters because a two-thrust planar quadrotor that loses a rotor entirely cannot be recovered by any controller. Faults switch on mid-episode at random times.

Crucially, each variant includes "out-of-prior" effects that the offline training never saw. These are asymmetric drag from a damaged blade, effectiveness that varies with commanded thrust, and simultaneous two-actuator loss. These effects are what $h_t$ must learn online.

As a higher-dimensional secondary testbed, we use the Ant velocity task from Safety Gymnasium with the per-joint effectiveness coefficient $k$ from the curriculum paper. There the model predictive controller is replaced by a sampling-based planner (one that tries many candidate action sequences and keeps the best) over the learned model.

The second part trains $g_\phi$ offline on simulated transitions across sampled $\theta$, using the true $\theta$. It then fine-tunes $g_\phi$ on estimator outputs, so that the model sees the noisy $\hat\theta$ it will actually receive. Separately, we characterize $\sigma_\theta$ for each estimator and estimate $L_\theta$ empirically.

The third part runs online deployment and analysis. We measure how many post-fault samples each method needs to bring tracking error within a fixed tolerance. We measure constraint violations during the post-fault window. We also measure Expected Maximum Consecutive Cost steps (EMCC) from Revisiting Safe Exploration in Safe Reinforcement learning. EMCC counts the longest run of consecutive violations, which separates a scattering of minor infractions from a sustained excursion toward a crash. Average cost alone hides that difference.

Baselines

The naive reference is a fixed-gain controller tuned for the healthy plant, plus a PPO agent with a large crash penalty. The penalized PPO agent represents the reward-shaping approach that Quanyi Li et al. show either sacrifices safety or freezes. These anchor what "no fault tolerance" and "naive learning" cost. Mao et al. report Z-axis tracking error of 0.09 m for the non-adaptive controller under 40% propeller damage, against 0.003 m with adaptation.

The strongest model-based competitor is GP-MPC from safe-control-gym, starting from the healthy nominal model. It is the strongest competitor because it is already the most sample-efficient safe learner in this benchmark. The only reported results are about 1/10th the data of PPO/SAC and correction of up to 50% model error within seconds. There is no reported result found on Lacuna for GP-MPC under actuator faults.

For fairness, GP-MPC will use the same nominal $f_0$, the same kernel, the same horizon and solver, and the same online data budget as our method. Its constraint tightening will use the same $\beta$, tuned with the same grid on held-out fault seeds. Beating it means reaching the tracking tolerance with at least three times fewer post-fault samples, while showing no more violations and lower EMCC.

The strongest model-based fault-tolerant competitor is observer-plus-NMPC: the NMPC design using $\hat\theta$ from the same estimator, with no learning. That paper reports a 100% recovery rate against 85% for a cascaded controller in randomized simulation, though on its own platform rather than ours.

The strongest learning-based competitor is a domain-randomized teacher-student policy in the style of Saving the Limping. That paper reports 100% real-world survival under single locked joints and 45.8% survival in simulated discrete terrain. This policy gets the same simulator, fault distribution, and wall-clock training budget as our offline stage. To test whether pure constraint enforcement suffices, we add a PPO-Lagrangian version of it. A Lagrangian method adds the constraint to the objective with a learned weight. Safety Gymnasium reports that PPO-Lagrangian achieves a 98% reduction in violations over PPO, with a 45% drop in reward.

Finally, SafeDreamer serves as the world-model baseline. It is trained healthy and then fine-tuned online after the fault with the same sample budget.

Three ablations isolate the key ingredient. The first freezes $g_\phi$ at the healthy $\theta$, so only the Gaussian process adapts; this is essentially GP-MPC with a better prior. The second removes $h_t$, leaving only the indexed prior. The third sets $\sigma_\theta = 0$ in the margin, which tests whether estimator uncertainty is what prevents violations in the first 0.2 seconds after a fault.

Why not just…?

Why not just plug the fault estimate into an NMPC and skip learning? This is the real threat. If the fault is fully described by effectiveness multipliers, observer-plus-NMPC should match our method, because $g_\phi$ adds nothing a correct physics model lacks. The NMPC paper's own need for an extra inversion layer suggests that damaged-flight effects exceed the model, but that is indirect evidence. The experiment settles it. On faults that the multipliers fully describe, we expect a tie and will report it. On out-of-prior faults, our method must reduce tracking error and violations below observer-plus-NMPC. If it does not, the learned components are not worth their cost, and the honest conclusion is "estimator plus physics is enough."

Why not just run GP-MPC, which already learns residuals with little data? GP-MPC corrects errors of up to 50% within seconds. A seized actuator, however, is a near-100% error on one input channel, and it occurs while the robot is moving. Starting from the healthy model, the Gaussian process must learn that large error from data collected while the margins are wide and the controller is conservative. This is exactly the starvation pattern described in the safety-paradox paper. The frozen-$g_\phi$ ablation and the head-to-head sample counts measure this directly. If GP-MPC recovers in comparable time, our core claim fails.

Why not just randomize every fault in simulation and deploy a single policy? For faults inside the training distribution this needs no online samples, and on those we expect it to be competitive. It enforces no constraint while learning. More importantly, it cannot correct itself on held-out fault modes such as two-actuator loss or thrust-dependent degradation, because it has no online learning channel. We test on exactly those held-out modes, and report in-distribution results as well, where it may win.

Why not just fine-tune a world model like SafeDreamer after the fault? Its model has no fault input, so the first post-fault planning steps use healthy dynamics, and fine-tuning starts only after errors accumulate. Its online planning is also expensive. We measure violations and EMCC in the first second after the fault, which is where this weakness should show.

Proposed Experiments

Each method is run over 50 fault-injection seeds per fault type. Faults appear at random times. We record five quantities: post-fault samples to reach the tracking tolerance, violation count and EMCC in the post-fault window, recovery rate, and tracking RMSE after recovery.

Three outcomes are decisive. First, on out-of-prior faults, our method must beat both GP-MPC (on samples to recover) and observer-plus-NMPC (on error and violations). Second, the $\sigma_\theta = 0$ ablation must show violations concentrated in the first 0.2 seconds after the fault, the window in which the estimator is still converging. If it does not, estimator uncertainty is not what the margin is protecting against. Third, on the over-actuated variant, our method must drive the healthy thrusters to saturation while keeping state violations at zero, whereas a constrained learner with actuator-usage margins must fail. That would confirm that putting the margin on state rather than effort is what lets degraded systems survive.

A mismatch could also appear: the learned prior might generalize poorly from true $\theta$ to noisy $\hat\theta$. To check this, we will report results with true $\theta$ supplied and with $\hat\theta$ supplied. The gap between the two quantifies how much of the method's value depends on the estimator.