Earned Compliance: Online Feedback-Alignment Learning of Robot Dynamics Inside the Impedance Loop

A robot arm that is soft in how it responds to contact (low "stiffness," the virtual spring constant its controller applies to position errors) is safe around people and gentle on parts. It can only stay accurate, though, if it has a good internal model of its own dynamics that predicts the torques its motors need. Learned hybrid models that combine textbook rigid-body physics with a small recurrent network already make low-stiffness tracking accurate. They are trained offline with backpropagation through time, so they cannot keep up when the robot picks up an unknown payload or its joint friction changes mid-task. This note proposes updating the model's neural part at control rate using the error signal the impedance controller already computes. The controller's corrective torque is broadcast directly to every layer, as in direct feedback alignment, so no backward pass through the network or through time is needed. The robot's stiffness is then lowered only as fast as the online model earns it.

The idea in two sentences: the impedance controller's feedback torque is an error vector already expressed in the model's output space, so it can train the dynamics model layer-by-layer, 1000 times a second, without backpropagation. Because stiffness scales that error signal and also decides how much model error matters, stiffness and learning should be scheduled together. The robot stays stiffer while its model is wrong and becomes compliant as the model converges.

Core Claim

The closest system is End-to-End Learning of Hybrid Inverse Dynamics Models for Precise and Compliant Impedance Control. It fuses rigid-body dynamics, a friction term, and an LSTM residual on a Franka Panda, with a Cholesky-based parameterization ("DiffBary") that keeps learned inertias physically valid. It reaches torque prediction NMSE of 0.0033, against 0.0081 for pure rigid-body dynamics. Its authors state that recurrent backpropagation is too heavy for the 1 kHz loop, so the model is frozen at deployment. We keep their model class and replace only the learning rule used at deployment. The claim is that a direct-feedback-alignment update driven by the controller's feedback torque, started from feedback matrices matched to the pretrained weights, recovers most of the accuracy lost after a dynamics shift within seconds. The claim covers shifts that change the network's hidden features and not just its output scale, such as friction and stick-slip changes. Tying stiffness to the online model's residual should then cut contact-transition force spikes relative to either fixed stiffness or error-driven heuristic stiffness laws.

Concretely, let $e_t = \tau^{fb}_t - \tau^{ext}_t$ be the impedance controller's feedback torque minus the external torque estimated from the wrist force/torque sensor. This is the torque the feed-forward model failed to supply. Each hidden layer $l$ of the residual network is updated with $$\delta h_l = (B_l e_t) \odot \sigma'(z_l),$$ where $h_l$ is the layer's activation, $z_l$ its pre-activation, $\sigma'$ the activation derivative, and $B_l$ a fixed feedback matrix. The update uses only the current time step's activations. Nothing is unrolled through time, and every layer updates in parallel. Stiffness is scheduled from a running estimate $\hat{r}_t$ of the model's torque residual: $$K_t = K_{\min} + (K_{\max} - K_{\min})\,\min(1, \hat{r}_t / r_{\text{ref}}),$$ where $K_{\min}$ and $K_{\max}$ are the allowed stiffness range and $r_{\text{ref}}$ is the residual at which full stiffness is used. The rule makes a simple point: compliance is safe exactly when the feed-forward model is accurate. It is also consistent with the learning signal, because higher stiffness gives larger, higher signal-to-noise feedback torques while the model is wrong.

Research Program

The method borrows from three sources. The update rule is Direct Feedback Alignment Provides Learning in Deep Neural Networks, which projects the output error to each layer through fixed random matrices and learns as long as the projected signal stays within 90° of the true gradient. Its stability tricks come from Direct Feedback Alignment Scales to Modern Deep Learning Tasks and Architectures, which showed that optimizer settings such as Adam's second-moment decay keep non-gradient updates stable. Initialization is the critical ingredient. Random $B_l$ would destroy the offline model's accuracy at the moment it is most needed. Enabling Fine-Tuning of Direct Feedback Alignment via Feedback-Weight Matching showed that plain DFA fails on pretrained weights for exactly this reason. Building feedback matrices from an SVD of the pretrained weights restores alignment: 82.67% versus 74.70% accuracy on CIFAR-100→SVHN, and STSB correlation of 0.76 versus 0.10 with BERT. We apply that matching to the offline-trained hybrid model before deployment.

The recurrence needs care. We treat the LSTM's hidden state as an input at each step and update input-to-gate and output weights with the broadcast error, forgoing credit assignment through time. That is the main approximation, and we measure its cost directly. Physical parameters are handled separately. Rigid-body torque is linear in the barycentric parameters, so we update them by exact recursive least squares projected onto the DiffBary-valid set. The novelty question therefore concentrates entirely on the neural residual.

On the analysis side, we will log the angle between the DFA update and the true truncated-backprop gradient at every step, across shift types. This tests whether the alignment reported for image classification holds for a torque-regression network under distribution shift. The file warns of the contrary case. DFA loses ground on precision regression (NeRF PSNR 25.41 versus 31.01 dB), and synthetic gradients tripled rotation error from 3.45° to 12.65° in Feed-Forward On-Edge Fine-Tuning Using Static Synthetic Gradient Modules. That paper is the only prior work putting backprop-free updates on a robot, and only for vision-based grasp prediction. An imprecise online update could corrupt feed-forward torque, which is why the stiffness schedule uses the model's measured residual as its gate and not its update magnitude. If adaptation hurts, stiffness rises instead of the robot trusting a bad model.

Baselines

The naive reference is the frozen offline hybrid model from End-to-End Learning of Hybrid Inverse Dynamics Models for Precise and Compliant Impedance Control, run at fixed low stiffness. Its reported number is the target to preserve before any shift, torque NMSE 0.0033 on its Franka dataset. The same model at fixed high stiffness serves as the "just be stiff" reference for contact forces.

The strongest classical competitor for adaptation is recursive least squares on the physical parameters plus the residual network's final linear layer, with hidden features frozen. It is exact, cheap, and runs at 1 kHz. It is the most likely method to make the proposal unnecessary. It gets the same data stream, the same compute budget, and a forgetting factor tuned with the same number of trials as our learning rate.

The strongest learning-based competitor is truncated backpropagation through time with window $k$, run on the same hardware. $k$ is chosen as the largest window that fits the 1 kHz budget, plus an off-loop variant that updates at a lower rate. Beating it means lower post-shift NMSE at equal wall-clock and memory.

For stiffness, the competitors are the muscle-inspired law $\dot{K} = \beta|\epsilon| - \gamma E$ from Impedance Primitive-augmented Hierarchical Reinforcement Learning for Sequential Tasks, with its $\beta,\gamma$ tuned on the same budget, and the control-theoretic gain optimizer of Safe Online Gain Optimization for Variable Impedance Control. The latter reports 1.39–2.50 s settling and 0.09–0.14 N² steady-state force variance on unknown plastic, metal, and wood, versus 10.14–16.54 s and 2.21–7.41 N² for adaptive variable impedance control.

Two ablations isolate the key ingredients. One uses random feedback matrices in place of weight-matched ones, which tests initialization. The other decouples the stiffness schedule from learning, using adaptation with fixed stiffness or the schedule with a frozen model.

Why not just…?

Why not just adapt the physical parameters and the last layer with recursive least squares? This is the real threat. Classical adaptive control exploits exactly this linearity. If payload changes are the only shifts, it will likely match or beat our method, because a payload is a physical-parameter change. Our bet is that friction, stick-slip, and joint-flexibility changes alter the features the recurrent residual computes, and last-layer RLS cannot reshape features. The experiments separate payload shifts from friction and temperature shifts. If RLS matches on both, we will report that deep online adaptation is unnecessary and redirect the work to the stiffness schedule alone.

Why not just run truncated backprop through a small network? For a small LSTM, a short truncated backward pass may fit in 1 ms. The advantages of direct broadcast are depth-independent update time and no stored activations, and they grow with model size. We compare at matched wall-clock and also scale the residual network's depth. The honest prediction is that truncated backprop wins at small sizes. The claim holds only if a crossover appears within model sizes that actually improve accuracy.

Why not just stay stiff until the model is good, or use an analytical gain law? Fixed high stiffness is the safety failure the whole field is avoiding. The rigid MAPLE baseline was judged too hazardous to test on hardware in the IMP-HRL study. Heuristic laws need per-task tuning of $\beta,\gamma$. The control-theoretic optimizer adapts quickly but has no model of the arm's own dynamics to improve. Our schedule differs because it keys stiffness to model accuracy rather than tracking error, and the contact-transition experiments below test whether that distinction reduces force spikes.

Proposed Experiments

The first testbed is the Franka tracking dataset from the hybrid-model paper, replayed with synthetic shifts: added end-effector mass, scaled viscous and Coulomb friction, and injected stick-slip. We report torque NMSE over time after the shift, time to recover within 1.5× the pre-shift 0.0033, and the per-layer alignment angle. The same protocol then runs on a physical Franka that grasps unknown payloads mid-trajectory and runs after thermal warm-up changes friction. There we measure tracking error under low stiffness, since the source paper reports only that baseline errors "skyrocket," with no millimetre figure.

For contact, we reproduce the material-transition protocol of the gain-optimization paper, contacting plastic, metal, and wood, and report settling time, steady-state force variance, and peak transition force against its reported ranges. In robosuite, we plug the adapted dynamics model and stiffness schedule into the IMP-HRL Wipe and Door tasks with randomized payload and friction. The reference is the 86% Wipe success reported for IMP-HRL, against 42% for rigid MAPLE.

Three results would settle the question. The proposal fails if last-layer RLS matches it on friction shifts. It fails if truncated backprop matches it at every model size fitting the 1 kHz budget. It also fails if random-feedback DFA performs as well as weight-matched DFA, which would mean the prior-work mechanism we rely on is irrelevant here. It succeeds if weight-matched broadcast learning recovers post-shift NMSE faster than both on feature-level shifts, and the model-gated stiffness schedule lowers peak contact force relative to the muscle-inspired law at equal task success.