Robots learn most manipulation skills by imitation, meaning supervised learning on recorded human demonstrations. Those demonstrations are almost always slow, because slow motion is easy to teleoperate and safe to record. Physically, slow motion is quasi-static: inertia barely matters, objects stop when the hand stops, and the geometry of a good trajectory does not depend on how fast it is played. Deployment often needs the opposite. Belts move, objects are tossed, and cycle time matters. Today a policy trained slowly either fails when sped up, or it needs many hard-to-collect fast demonstrations. We propose to treat execution speed the way neural combinatorial optimization treats problem size, as a scale axis. Policies trained at small scale are known to collapse when the scale grows, unless the representation removes the part of the problem that does not change with scale.
The core idea fits in two sentences. In the quasi-static regime, a task is exactly invariant to slowing down or speeding up time, so a policy built from leaky integrators can absorb that invariance by tying its time constants to the commanded speed. What remains when speed increases is a physics residual whose size grows with a single dimensionless ratio of inertial to frictional (or reaction-time to tolerance) effects, so the residual can be fitted from a handful of medium-speed demonstrations and extrapolated to speeds never demonstrated.
Core Claim
The cross-scale generalization literature shows a consistent pattern: zero-shot transfer to larger instances works when the policy's state is reformulated so that scale-changing information is removed. BQ-NCO trains an imitation policy on 100-node routing problems. It represents only the remaining "tail subproblem" and groups equivalent partial solutions, and it reaches roughly a 1.38% optimality gap at 1,000 nodes where other neural solvers sit at 20–40%. Purity Law finds a structural statistic of optimal tours that holds across scales and builds a reward around it. Both papers supply our method template. Neither touches continuous physics.
We claim that for manipulation, speed has the same structure and an even cleaner invariant. The quasi-static model is a deliberately simplified model of the physics. Within its regime, a demonstration played at speed factor $s$ is still a valid demonstration. Outside that regime, the error of the simplified model is not arbitrary. To leading order it scales with a dimensionless group, for example $\Pi = v^2/(\mu g L)$ for sliding, where $v$ is a characteristic speed, $\mu$ the friction coefficient, $g$ gravity and $L$ the task's spatial tolerance. This form comes from comparing the kinetic energy shed by friction, $v^2/(2\mu g)$, against the tolerance $L$.
The policy therefore decomposes as
$$ a_t = \pi_{\text{qs}}\big(o_{t};\, s\big) + \sum_{k=1}^{K} \Pi^k\, r_k(o_t), $$
where $a_t$ is the action, $o_t$ the observation, $\pi_{\text{qs}}$ a speed-equivariant quasi-static policy, and $r_k$ small learned residual heads with $K \le 2$. What is new is a structured, physically scaled extrapolation in speed, learned from imitation data that is mostly slow. The novelty is not a dynamic-manipulation trick for one task.
Why a leaky-integrator trunk
The simplified-neuron direction supplies the trunk, and the reason is mechanical rather than biological. A leaky integrator, the deterministic core of the leaky integrate-and-fire neuron, obeys $\tau\, dV/dt = -V + I(t)$, with membrane state $V$, input $I$ and time constant $\tau$. If the input is time-warped, $I(t) = J(st)$, then setting $\tau' = \tau/s$ gives exactly $V'(t) = V(st)$.
A recurrent policy whose every time constant is divided by the commanded speed is therefore exactly equivariant to time warping. It does not need to see sped-up data to behave correctly in the quasi-static regime. This is the speed analogue of BQ-NCO's quotienting: states that differ only by a time warp are mapped to the same internal trajectory.
A second benefit is that such trunks are tiny and cheap. SpikePingpong shows that a small learned residual over a ballistic physics model, fed by a 20 kHz spike camera, runs at 0.407 ms and intercepts 92% of balls, where Diffusion Policy manages 6% and ACT 19%. We borrow its residual-over-physics structure. We differ in three ways: we place the simplified dynamics inside the policy rather than at the sensor, we make the residual explicitly a function of a speed-dependent dimensionless group, and we test extrapolation in speed, which SpikePingpong never varies. Gated and adaptive LIF variants (GLIF, DA-LIF) are candidate cells. We will use non-spiking leaky integrators by default and treat spiking as an optional latency ablation, because spiking is not needed for the equivariance argument.
Relation to work that already bridges slow and fast
Two systems connect quasi-static motion and imitation learning, and both motivate this proposal by leaving the switching between regimes hand-designed. DynamicManip turns one static demonstration into dynamic training data. It reconstructs object meshes and edits trajectories over sampled object velocities, then deploys a diffusion policy with a learned stage classifier, and it beats 3D Diffusion Policy by 18.4 percentage points. Its augmentation covers the velocity range it samples. We ask what happens beyond that range, and we replace per-velocity trajectory editing with a policy whose speed dependence is constrained by physics. BILBO opens bags with a dynamic movement primitive followed by quasi-static refinement, switching at fixed geometric thresholds. In our formulation that switch becomes the smooth growth of $\Pi$.
HYDRA splits actions into sparse waypoints and dense contact actions but stays kinematic. Mixture of Horizons fuses action chunks of several lengths and uses cross-horizon agreement to decide when to replan. That is a temporal-scale idea, but it varies the planning horizon, not the physical speed, and it is evaluated on slow LIBERO tasks.
Two papers act as constraints. The quasi-static LCQP planner of Katayama et al. drops from 100% to 48–60% pivoting success under model and sensor noise. That warns that $\pi_{\text{qs}}$ cannot be a pure physics solver, so we learn it from demonstrations. Song & Boularias report that high-speed slides produce unmodeled residual motion. That is exactly the residual we parameterize, and their differentiable sliding simulator gives a controlled testbed.
Research Program
We will build the policy in three parts.
The first part is a leaky-integrator recurrent trunk over visual and proprioceptive features, with every time constant set to $\tau_i/s$. It is trained by behavioral cloning on slow demonstrations alone.
The second part is a residual head with polynomial dependence on $\Pi$. It is trained on a small set of demonstrations collected at one or two intermediate speeds. In simulation the fast demonstrations come from scripted experts; on hardware they come from DynamicManip-style edited trajectories.
The third part is a per-task derivation of $\Pi$ from the dominant neglected force. For belt picking, $\Pi$ is belt speed times controller latency over grasp tolerance. For sliding, it is the friction ratio above. For high-acceleration arm motion, the neglected forces are centrifugal and Coriolis, following the failure mode noted by Osumi et al..
The analysis has two parts. We will measure how well each policy's error versus speed matches the predicted low-order scaling in $\Pi$. We will also test whether replacing the derived $\Pi$ with raw speed, or with a learned scalar, destroys extrapolation. If a learned scalar does equally well, the physics derivation is unnecessary and we will report that.
Baselines
The naive reference point is time-resampling: train a standard behavioral-cloning policy on slow demonstrations and replay its actions with timestamps compressed by $s$. In the quasi-static regime this is already correct for open-loop motion. Any gain we report must therefore come from closed-loop behavior at higher speed or from the dynamic regime, and this baseline sets that floor.
The strongest existing competitor is DynamicManip's pipeline. This is a diffusion policy trained on velocity-augmented edited trajectories with adaptive chunking. It is the only published system that turns static demonstrations into dynamic skills on our benchmark. The file gives only its relative gain (+18.4 points over DP3 across its tasks) and no absolute per-task success rates, and it has no reported result found on Lacuna for extrapolation beyond its sampled velocity range. We will rerun it from the same slow demonstrations plus the same medium-speed demonstrations we give our method, with velocity augmentation restricted to the training speed range, matched wall-clock training compute, and an equal hyperparameter search budget of 20 configurations per method. Beating it means higher success at speeds at least twice the fastest training speed, without losing more than 3 points inside the training range.
A speed-conditioned transformer receives the same data with $s$ (or $\Pi$) as an input token. It tests whether conditioning alone extrapolates. For latency, ACT and Diffusion Policy serve as references, using SpikePingpong's in-setup numbers of 19% and 6% success against 92% for the residual system as evidence that heavy policies fail under tight reaction windows.
Two ablations isolate the key ingredients. In the first, our trunk keeps fixed rather than speed-scaled time constants, which removes the equivariance. In the second, the residual head takes raw speed instead of $\Pi$, which removes the physical scaling. On slow LIBERO execution, the reference ceiling is Mixture of Horizons at a reported 99% average success with $\pi_{0.5}$. We use it only to confirm that our policy does not degrade slow-speed performance.
Why not just…?
Why not just augment with sped-up trajectories, as DynamicManip does? Augmentation teaches the policy the speeds it sees, and nothing in a diffusion policy constrains how it behaves outside that range. This is a real threat inside the training range, where augmentation may match or beat us. The decisive test is the out-of-range sweep. If augmentation extrapolates as well as our policy, the structured residual is not needed and we will say so.
Why not just condition a standard policy on speed? Conditioning lets a network interpolate between seen speeds, but cross-scale results in routing show that unstructured models degrade badly beyond the training scale, with 20–40% gaps at 10× size in BQ-NCO's comparison. The speed-conditioned transformer baseline and the raw-speed ablation test exactly this.
Why not just slow the robot down and use a quasi-static planner? For many tasks this is acceptable. But belt picking and catching have no slow version, and quasi-static planners are brittle under model error, falling to 48–60% in Katayama et al.. We include tasks where the required speed is set by the environment, not chosen by the robot.
Why not just resample time on an ordinary recurrent network or transformer instead of using leaky integrators? In the quasi-static regime, time-normalized inputs to a standard network may give the same equivariance. This is the most likely place the simplified-neuron component turns out to be optional. The fixed-time-constant ablation, paired with a time-normalized transformer, settles it. If they tie, the trunk's remaining justification is latency alone, measured directly.
Proposed Experiments
The first testbed is RoboTwin 2.0 (RoboTwin 2.0) with DynamicManip's simulated tasks. Belt Picking and Object Catching have a natural speed knob. We will train with slow demonstrations at belt or object speeds in the lowest quarter of the range, plus 10 demonstrations at one mid speed. We will then test on a sweep reaching four times the fastest training speed.
The second testbed is planar sliding in the differentiable simulator of Song & Boularias. There we vary push and release speed and measure final-position error against their reported 1.53 cm simulated prediction error at slow speeds.
The third testbed is LIBERO-Spatial and LIBERO-Long (LIBERO) executed at 1×, 2× and 4× commanded speed. This tests whether slow-trained skills survive when sped up.
Metrics are success rate as a function of speed and the speed at which success halves. On sliding we also report position error. We report policy latency as well.
Three outcomes decide the proposal. The first is whether our policy's half-success speed exceeds the augmented diffusion baseline's by a clear margin outside the training range. The second is whether the residual magnitude measured on held-out speeds follows the predicted polynomial in $\Pi$. The third is whether the derived $\Pi$ beats raw speed in the residual head. If the first holds but the third does not, the contribution is the decomposition rather than the physics. If neither holds, the speed-as-scale analogy fails for manipulation.
References
- BQ-NCO: Bisimulation Quotienting for Efficient Neural Combinatorial Optimization
- Purity Law for Neural Routing Problem Solvers with Enhanced Generalizability
- SpikePingpong: Spike Vision-based Fast-Slow Pingpong Robot System
- GLIF: A Unified Gated Leaky Integrate-and-Fire Neuron for Spiking Neural Networks
- DA-LIF: Dual Adaptive Leaky Integrate-and-Fire Model for Deep Spiking Neural Networks
- DynamicManip: Enabling Dynamic Manipulation from a Single Static Demonstration
- Dynamic Manipulation of Deformable Objects using Imitation Learning with Adaptation to Hardware Constraints
- HYDRA: Hybrid Robot Actions for Imitation Learning
- Mixture of Horizons in Action Chunking
- Quasistatic contact-rich manipulation via linear complementarity quadratic programming
- Learning to Slide Unknown Objects with Differentiable Physics Simulations
- Development of a Cooperative System Comprised of a Robot and a 7 D.O.F. Crane with Three Wires.
- RoboTwin 2.0: A Scalable Data Generator and Benchmark with Strong Domain Randomization for Robust Bimanual Robotic Manipulation
- LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learning