Many robots and wearable devices already store the energy they absorb. A leg that brakes during a landing can charge a spring or push current back into its battery, and a stair tread can bank a descending person's weight for their next ascent. What these systems still lack is a good decision about when to spend that stored energy. Current devices release it by fixed rules, such as "unlatch when the trailing foot unloads," or by trajectories computed offline. These rules stop working as soon as the user, the load, the terrain, or the next task changes. We propose treating the release decision as an online prediction problem. After every storage-and-release cycle, the device updates a small Bayesian model of how much actuator work a release at a given moment would save, and it uses that model's uncertainty to choose the next release time.
The model has to learn from a handful of cycles, run inside a millisecond-scale control loop, and cope with dynamics that jump abruptly at foot contact and at latch engagement. To meet all three demands we use random Fourier features, an explicit randomized feature map whose inner products approximate a kernel similarity, so that Gaussian-process-style regression becomes ordinary Bayesian linear regression with a fixed-size update. We make the features phase-aware: each contact phase gets its own block of features, so the model does not blur across impact boundaries. The claim is narrow and testable. On devices with a controllable elastic buffer, an uncertainty-guided, online-updated release policy should recover more of the stored energy after a task or user change than fixed release rules, and it should get there in far fewer cycles than a reinforcement-learning policy trained offline with an energy-aware reward.
Core Claim and What Is New
The energy-recycling literature supplies the hardware setting, and it motivates the gap. Ha et al.'s energy-recycling assistive stairs lock spring-loaded treads during descent with an electromagnetic latch and release them during ascent. With a fixed, sensor-triggered release, they report a 37.7 ± 10.5% reduction in knee positive work. Ambrose and Ames's parallel-elastic hopper gains 2.5× mechanical efficiency through open-loop playback of a precomputed trajectory. Li et al.'s hip-only exoskeleton uses an electromagnetic clutch-spring at the knee. All three rely on schedules fixed in advance. The research file names the absence of online predictive models that adapt absorption and release schedules to contact timing and task switches as an open problem. Here "inter-task" has a specific meaning: energy is captured in one task, such as descent or landing, and spent in a different task, such as ascent or take-off. The best release moment therefore depends on what comes next, not only on the current state.
Random-feature learning supplies the method. Bayesian RL for Goal-Only Rewards already showed that random Fourier features make Bayesian value learning cheap enough for continuous control, at a per-step cost that depends only on the number of features. It also reports that performance drops to 60% success on Hopper and that its stationary kernel breaks down at contact. Inferring Smooth Control used Gaussian-process priors with random features to smooth action sequences. It never models energy that could be recovered. PEGrad treats energy purely as a cost to push down. None of these works predicts the value of releasing banked energy. None of them handles a discrete latch event inside the kernel model either.
Our contribution is to shrink the decision to a low-dimensional, per-cycle choice of when to release and how much, which is where the kernel machinery is actually useful. We then pair that choice with a piecewise-stationary random feature map, and we test both on hybrid-dynamics energy buffers.
Research Program
The learner works in cycles. Each cycle $c$ is one storage-and-release event. At release time the device observes a context vector $x_c$. This vector contains the buffer's stored energy (from spring deflection or buffer state of charge), the current contact phase, joint velocities, and a short summary of the preceding task segment. The learner then chooses a release parameter $a_c$: a release delay relative to a contact event, together with a clutch-engagement fraction. After the positive-work phase it measures the outcome $y_c$, defined as the actuator positive work saved relative to a no-release reference on the same segment. The model is
$$y_c = w^\top z_{p(c)}(x_c, a_c) + \epsilon_c, \qquad w \sim \mathcal{N}(0, \sigma_w^2 I),$$
where $z_p$ is a random Fourier feature map of dimension $m$. It is drawn separately for each contact phase $p$ (flight, stance, latched, unlatched), and it is zero outside phase $p$. The noise $\epsilon_c$ is Gaussian.
Splitting features by phase makes the implied kernel stationary within a phase and uncoupled across phases. This is the simplest way to stop the smooth kernel from averaging across an impact, a failure mode the file identifies for shift-invariant features. Because the model is linear in $w$, each cycle's posterior update is a rank-one update costing $O(m^2)$, independent of how many cycles have been seen. This constant cost is what separates it from an exact Gaussian process, whose cost grows with the data.
Actions are chosen by Thompson sampling from the posterior over $w$, restricted to release parameters that satisfy safety limits. To reduce the Monte Carlo variance of the feature draw, which the file flags as a way to destabilize control loops, we use orthogonal or structured feature draws. This follows the direction of Recycling Randomness with Structure. We choose $m$ empirically rather than from the uniform bound of Optimal Rates for Random Fourier Features. That bound's $m^{-1/2}$ rate tells us to expect diminishing returns beyond a few hundred features for a context of about ten dimensions, but it is too loose to set $m$ precisely.
The release learner sits on top of an existing low-level controller. It does not replace locomotion control. This separation is deliberate. It keeps the kernel model's input small, which is where random features are known to work, and it avoids the high-dimensional regime where the Hopper result already shows them failing.
We build three simulated testbeds in MuJoCo. The first is a vertically constrained moving-mass hopper with a series spring and a clutched parallel spring, modeled on Ambrose and Ames. Its tasks alternate between drops from varying heights and hops to commanded heights. The second is a planar two-joint leg with four-quadrant electrical recuperation, modeled on the cable-driven Capler-Leg. Its buffer is electrical, and the decision is how much regenerated charge to route into the next push-off. The third is a Unitree H1 humanoid from HumanoidBench, modified with an added clutch-spring at each knee. It performs stair descent-then-ascent sequences, the inter-task case ERAS targets. On all three we vary spring stiffness, payload, recuperation efficiency, and task order between deployment phases, so that adaptation can be measured directly.
Baselines
The naive reference is the same device with the clutch or latch never used, so that all positive work comes from the actuators. Beside it we run a fixed event-triggered release rule of the kind used by ERAS, with its trigger threshold grid-searched on the training condition. This baseline reflects current practice. The best fixed rule re-tuned for each test condition gives an oracle ceiling that tells us how much adaptation is worth at all.
The strongest competitor is a PPO policy trained with the same simulators and the same actuator. It outputs the same release parameters and receives a reward that credits recovered energy, meaning negative work fed into the buffer counts as income. It is trained with domain randomization over stiffness, payload, and task order. A second variant replaces the scalar reward with PEGrad's projected energy gradient. For context, PEGrad reports an average 64% energy reduction across DM-Control locomotion tasks. That figure concerns total-energy minimization on unmodified robots. We found no reported result on Lacuna for learned release timing of an elastic or electrical buffer on any of our testbeds, so the comparison has to be run from scratch. Both reinforcement-learning variants receive the same total simulator step budget that the kernel learner sees, including its offline prior-fitting data, and the same hyperparameter search budget.
We also run two model-based peers that share the Thompson-sampling decision rule and differ only in the predictor. One is an exact Gaussian process with the same kernel. The other is a bootstrapped ensemble of small neural networks.
The key ablation replaces the phase-split features with a single stationary random feature map over the same inputs, with the phase given as an input coordinate. A second ablation removes uncertainty and acts greedily on the posterior mean.
The proposal counts as beating the baselines only under a joint condition. After a condition change, it must reach a fixed fraction of the oracle's saved work in significantly fewer cycles than the domain-randomized PPO and PEGrad policies, which adapt only through whatever their observations carry. It must also do so without losing task success, and with per-cycle compute that fits a 1 kHz controller budget.
Why not just…?
Why not just train deep RL with a reward that credits regenerated energy? This is the most serious threat. A policy trained with domain randomization might carry enough context in its observations to release well under new stiffness or payload without any online learning. Evidence in the file only partly addresses this. Peri et al. show that scalar energy terms are brittle to tune, and MyoSuite reports that flat policy gradients need over 70 million samples on dexterous tasks. Neither paper tested release timing. We therefore make the adaptation experiment decisive. We include test conditions outside the randomization range, such as a spring 30% stiffer than anything seen in training, and we count cycles to recovery. If PPO matches the kernel learner in and out of range, the online-learning claim fails and we will report it.
Why not just use an exact Gaussian process? With a few hundred cycles, an exact or sparse Gaussian process is cheap enough, and it may predict slightly better. The random-feature version earns its place only through constant-cost updates over long deployments, on the order of thousands of cycles per user-day, and through a bounded memory footprint on embedded hardware. We compare prediction error and decision quality at matched wall-clock and memory budgets on a microcontroller-class CPU. If the exact process wins at every deployment length we test, we will recommend it.
Why not just re-tune the fixed release threshold for each user or condition? The per-condition oracle measures exactly this. If the gap between a single tuned rule and the oracle is small, adaptive release is not worth building, and that outcome is informative for device designers. We expect the gap to be large mainly when task order varies, because a rule keyed to one contact event cannot know whether an ascent or a level step comes next.
Why not just use a data-driven Hankel predictor as in Data-Driven Predictive Control? That work adapts to payloads from 8 seconds of data. Its fundamental lemma assumes linear time-invariant behavior, however, and the latch makes the dynamics switch. We include a Hankel predictor of saved work as an additional predictor in the model-based comparison, to test whether phase-split kernels actually improve on it.
Decisive Experiments
The central experiment is a deployment sequence on each testbed. It consists of a training condition followed by four held-out changes in stiffness, payload, recuperation efficiency, and task order. We report cycles to reach 80% of the oracle's saved positive work, cumulative saved work over 500 cycles, task success, peak actuator force, and per-cycle update time.
The phase-split ablation is judged by prediction error near contact and latch events against error far from them. If the stationary features show no extra error near boundaries, the non-stationarity argument is wrong for this reduced input.
A sweep over feature count from 32 to 1024, comparing independent and orthogonal draws, measures how much variance in the feature draw reaches the release decisions.
If the humanoid stair results hold, the follow-on step is a bench-top clutch-spring knee rig. On that rig the same learner would run on-device, where the constant-time update is the property that matters.
References
- Stair negotiation made easier using novel interactive energy-recycling assistive stairs
- Improved Performance on Moving-Mass Hopping Robots with Parallel Elasticity
- Design and Evaluation of a Hip-Only Actuated Lower Limb Exoskeleton for Lightweight Gait Assistance
- Bayesian RL for Goal-Only Rewards
- Inferring Smooth Control: Monte Carlo Posterior Policy Iteration with Gaussian Processes
- Non-conflicting Energy Minimization in Reinforcement Learning based Robot Control
- Recycling Randomness with Structure for Sublinear time Kernel Expansions
- Optimal Rates for Random Fourier Features
- Cable-Driven Actuation for Highly Dynamic Robotic Systems
- HumanoidBench: Simulated Humanoid Benchmark for Whole-Body Locomotion and Manipulation
- MyoSuite - A contact-rich simulation suite for musculoskeletal motor control
- Data-Driven Predictive Control for Robust Exoskeleton Locomotion