DRAP: Dynamics-Risk Adaptive Plasticity for Continual Learning in Physical AI

Oswaldo LudwigarXiv 2026 · arXiv

TL;DR

DRAP (Dynamics-Risk Adaptive Plasticity) is a new replay-free method for continual learning in robotics that prevents "catastrophic forgetting" by identifying and protecting the neural network components responsible for physically sensitive moments in a task. By weighting parameter importance with a Lyapunov-inspired dynamics-risk score, it successfully retains old skills while learning new ones without needing to store any previous training data.

What Problem Does This Paper Address?

In the world of "Physical AI"—think robot arms or autonomous drones—learning is a never-ending process. However, when you train a model on a new task (like "stacking blocks") after it has already learned an old one (like "opening a drawer"), it often suffers from catastrophic forgetting, completely overwriting the old skill.

Standard continual learning (CL) methods often treat all data points as equally important. But in robotics, some moments are more critical than others. A tiny error while a robot is just moving through empty space is trivial, but a tiny error during a delicate "insertion" phase can lead to a total task failure. These errors compound over time, moving the robot into states it has never seen before. The authors argue that CL for physical agents must be "physics-aware," protecting the specific "muscle memory" required for these high-risk trajectory regions.

Methods

The authors introduce Dynamics-Risk Adaptive Plasticity (DRAP). It operates in three main stages:

  1. Dynamics-Risk Estimation: For every step in an expert demonstration, the system fits a local linear model to the physics: $s_{t+1} \approx A_t s_t + B_t a_t + c_t$. It then calculates a "risk score" based on the largest singular value of the state-transition matrix $A_t$. If this value is high, it means a small perturbation in the robot's state will be amplified in the next step—marking a "dynamically sensitive" region.
  2. Risk-Weighted Importance: As the model learns a task, it tracks which "tensors" (groups of weights in the neural network) are most active during these high-risk moments. It calculates a cumulative importance score for each tensor, weighted by the dynamics-risk.
  3. Adaptive Plasticity: When moving to a new task, DRAP doesn't use a standard penalty. Instead, it directly modifies the AdamW optimizer. Tensors that were highly important for previous critical moments have their learning rates slashed. They become "less plastic," effectively locking in the previous knowledge while allowing less critical parts of the network to adapt to the new task.
Fig. 1. Dynamics-risk computation from expert demonstrations. Local linear dynamics are estimated over sliding windows of logged state-action transitions. The largest singular value σmax(At) defines a local one-step perturbation-amplification score, which is robustly normalized and stored as a sample-level risk weight aligned with the training dataset.

To make this efficient for large models like the 7-billion parameter OpenVLA, they use KappaTune to select a specific subnetwork for training based on the "condition number" of the weight matrices, focusing adaptation on the most "plastic" parts of the architecture.

Datasets

The method was evaluated using the LIBERO benchmark, a standard for lifelong robot learning. The researchers tested a sequential pipeline across three suites:

  • LIBERO-Spatial: Tasks involving spatial relationships.
  • LIBERO-Object: Tasks involving specific object manipulations.
  • LIBERO-Goal: Tasks defined by semantic goals.

The evaluation protocol is strictly "replay-free," meaning once the model moves from the Spatial suite to the Object suite, it never sees the Spatial data again.

Key Insights and Ideas

  • Physics as a Regularizer: The most striking idea is that we can use classical control theory (Lyapunov stability) to guide modern deep learning. By identifying where physics is "unstable," we can tell the neural network which weights are too important to change.
  • Identifying Critical Phases: The "Risk Trace" (see below) shows that the algorithm naturally assigns higher risk to phases like "contact/manipulation" and "insertion," where precision is paramount.
  • Optimizer-Level Protection: Unlike methods like EWC (Elastic Weight Consolidation) that add a penalty term to the loss function, DRAP works at the optimizer level. This "immediate protection" prevents even the very first gradient step of a new task from damaging old knowledge.
Example Risk Trace showing how risk peaks during critical manipulation and insertion phases.

Key Results

DRAP significantly outperformed standard fine-tuning and the popular EWC baseline in maintaining a balance between learning new tasks and remembering old ones.

  • Retention: After training on all three suites, standard fine-tuning essentially "forgot" the first suite (Spatial) entirely, with a success rate dropping to near 0%. DRAP maintained a 34% success rate on that same suite without ever seeing the data again.
  • Average Forgetting: Across the entire sequence, DRAP reduced the average forgetting rate to 31%, compared to 77% for standard fine-tuning and 63% for EWC.
  • Success-Plasticity Balance: DRAP achieved the highest "Average Success" (55%) across all observed tasks at the end of the training sequence, proving it doesn't just "lock" the model but allows for meaningful new learning.

Limitations and Future Work

The authors acknowledge several areas for growth:

  • Simplified Risk Model: The current risk estimator only looks at the state-transition matrix ($A_t$). Future versions should include "control-path sensitivity" ($B_t$) to see how much an action error specifically contributes to the risk.
  • Offline Requirement: Currently, the dynamics-risk is precomputed from expert trajectories. An "online" version that estimates risk during live training would be more flexible.
  • Longer Sequences: While three suites is a start, real-world robots might need to learn dozens or hundreds of tasks. Testing DRAP's limits over much longer sequences is a necessary next step.

Summary updated

Abstract

Continual learning in physical AI differs from conventional continual learning because small per-step policy errors can accumulate over a trajectory. Each deviation may move the agent outside the training distribution, increase subsequent prediction errors, and ultimately cause closed-loop task failure. This work introduces Dynamics-Risk Adaptive Plasticity (DRAP), a replay-free method that estimates the historical importance of adaptable model tensors from sample-level action gradients weighted by local trajectory-amplification risk. A ridge-regularized local linear model is fitted to expert state-action transitions, and the largest singular value of its state-transition matrix provides a policy-independent, Lyapunov-inspired estimate of one-step perturbation amplification. After each task, risk-weighted squared gradients are accumulated in a tensor-level importance memory. During subsequent tasks, DRAP uses normalized cumulative importance to scale the effective AdamW learning rate of each tensor, immediately reducing the plasticity of components that supported previous behavior at dynamically sensitive trajectory points. We instantiate DRAP on a fixed KappaTune-selected subnetwork of OpenVLA and evaluate replay-free sequential adaptation across the LIBERO Spatial, Object, and Goal suites. Compared with standard AdamW fine-tuning and EWC, the results show substantially improved retention of previously learned behaviors while preserving strong new-suite acquisition.
BibTeX
@misc{ludwig2026drap,
  author = {Oswaldo Ludwig},
  title = {{DRAP: Dynamics-Risk Adaptive Plasticity for Continual Learning in Physical AI}},
  year = {2026},
  eprint = {2611.05029},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.05029},
}
Figure