Residual Control for Fast Recovery from Dynamics Shifts

Nethmi Jayasinghe, Diana Gontero, Francesco Migliarba, Spencer T. Brown, Vinod K. Sangwan, Mark C. Hersam, Amit Ranjan TrivediarXiv 2026 · DOI · arXiv

TL;DR

Robots operating in the real world often face unexpected changes in their dynamics (like a damaged leg or a heavier payload). This paper introduces a "cerebellar-inspired" control system that allows robots to quickly recover from these shifts without needing to retrain their core control policy. This is achieved by adding a small, adaptive "residual" control signal that works alongside the main, frozen policy, significantly speeding up recovery time.

What Problem Does This Paper Address?

Imagine a robot performing a task, like walking or manipulating an object. It's been trained to do this perfectly under specific conditions. Now, imagine that mid-task, something changes: one of its motors starts to fail, its weight distribution shifts, or it encounters a slippery surface. These "dynamics shifts" can cause the robot's performance to plummet, leading to instability or task failure. While some control methods can guarantee that the robot won't completely fall apart (input-to-state stability), they don't guarantee a fast recovery. The challenge is to enable a robot to adapt and regain its task performance quickly, in real-time, without needing to stop, reset, or retrain its entire control system, which is often impractical or unsafe in real-world deployments.

Methods

The core idea is inspired by how the biological cerebellum works. Instead of modifying the main, pre-trained "motor program" (the frozen reinforcement learning policy), the system adds a parallel, adaptive "residual controller." This residual controller injects corrective actions on top of the nominal policy's output.

Key components of this approach include:

  • Frozen Nominal Policy: A standard reinforcement learning policy (like Soft Actor-Critic) is trained offline and remains fixed during deployment. This ensures the base stability of the robot.
  • Additive Residual Channel: The adaptive part of the controller generates a "residual" action that is added to the nominal policy's action. This residual doesn't alter the original policy's parameters.
  • Stability Alignment Gate (SAG): This is the crucial innovation. The SAG acts as a guardian, ensuring the residual actions are helpful and don't destabilize the system. It does this through four mechanisms:
  • Magnitude Constraints: The residual action is kept within a bounded limit to ensure it acts like a "disturbance" rather than a complete policy change.
  • Directional Coherence: The residual action is encouraged to align with the nominal policy's direction, preventing it from actively fighting the stabilizing control signals.
  • Performance-Conditioned Activation: The residual controller only becomes significantly active when performance degrades, avoiding unnecessary adjustments during normal operation.
  • Adaptive Gain Regulation: The influence of the residual controller is dynamically adjusted based on how well the robot is performing.
  • Transient-Sensitive Feature Encoding: The system uses a fixed, high-dimensional non-linear expansion of state inputs, followed by a band-pass filtering mechanism. This helps the residual controller focus on the changes in behavior caused by the dynamics shift, rather than the steady-state behavior, leading to faster detection and correction.
  • Dual-Timescale Residual Generator: The residual action is generated by two parallel "heads" – a fast one for immediate, high-gain correction and a slow one for integrating persistent changes and stabilizing the system over time. This mimics the fast and slow learning processes observed in biological systems.
  • Online Error-Driven Plasticity: The residual controller's parameters are updated online based on the tracking error of the robot's performance. The learning rate is modulated by a task-level error proxy, allowing for faster learning when performance is poor and slower learning as it recovers.

Datasets

The paper primarily uses simulated robotic platforms for evaluation. These include:

  • Unitree Gol (Quadruped): Trained and evaluated for forward locomotion.
  • Agility Cassie (Biped): Evaluated for its robustness.
  • Unitree H1 (Humanoid): Evaluated for its whole-body control capabilities.
  • Agilex Scout Mini Pro (Wheeled Platform): Used for physical demonstrations.

The paper does not introduce new datasets but rather uses these established robotic simulation environments to test its method under various simulated dynamics shifts.

Key Insights and Ideas

  • Biological Inspiration Pays Off: Mimicking the cerebellum's architecture—separating a fixed baseline controller from a parallel, adaptive corrective module—is a highly effective strategy for real-time adaptation.
  • Bounded Residuals are Key: The "Stability Alignment Gate" is critical. Simply adding an adaptive signal can destabilize the system. Constraining its magnitude, direction, and activation based on performance is what preserves stability while enabling adaptation.
  • Focusing on Transients: The method's ability to quickly detect and react to changes in behavior (transients) rather than just the current state is a major driver of its fast recovery.
  • Separation of Concerns: Decoupling the nominal stabilization policy from the adaptation mechanism simplifies the problem and allows for more robust and generalizable solutions. The residual channel can be applied without needing to understand the specific dynamics shift.
  • Structural Constraints Over Representational Capacity: The ablation study highlights that the constraints (like directional coherence) are more important for stability and recovery than having a highly complex residual representation.

Key Results

The proposed method consistently demonstrated significantly faster recovery times across various robotic platforms and perturbation types compared to baselines:

  • Gol Quadruped: Recovery time reduced by up to 87% compared to a frozen SAC policy. For example, under mass increase (1.15x), recovery took 168 steps, compared to thousands for many baselines.
  • Cassie Biped: Recovery time reduced by 48%.
  • H1 Humanoid: Recovery time reduced by 30%.
  • Scout Wheeled Platform: Recovery time reduced by 20%.
  • Steady-State Performance: The method maintained near-nominal steady-state performance, often exceeding it (SSR > 1.0), indicating that rapid adaptation did not compromise long-term behavior.
  • Generalization: The approach showed consistent performance improvements across different robot morphologies (bipedal, quadrupedal, humanoid, wheeled) and perturbation types (actuator degradation, mass variation, friction changes).

Limitations and Future Work

  • Local Robustness Assumption: The method relies on the assumption that the nominal policy is locally stabilizing within a certain neighborhood of dynamics. It does not guarantee recovery from extremely drastic shifts that push the robot far outside this region.
  • No System Identification: The method does not explicitly identify the unknown dynamics. While this is a strength for generality, it means the adaptation is purely reactive to observed errors.
  • Future Work: The authors suggest exploring more sophisticated ways to modulate the residual authority, potentially incorporating more explicit task-level feedback or exploring how this approach could be combined with other adaptation strategies. Further investigation into the theoretical guarantees of stability under these residual corrections would also be valuable.

Summary updated

BibTeX
@misc{jayasinghe2026residual,
  author = {Nethmi Jayasinghe and Diana Gontero and Francesco Migliarba and Spencer T. Brown and Vinod K. Sangwan and Mark C. Hersam and Amit Ranjan Trivedi},
  title = {{Residual Control for Fast Recovery from Dynamics Shifts}},
  year = {2026},
  eprint = {2603.07775},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2603.07775},
}

Loading related content...

Figure