Impulse-and-Release Tokens for Energy-Efficient Humanoid Walking

Humanoid robots usually walk by stiffly tracking a joint-angle trajectory at every instant, so their motors spend energy fighting gravity and momentum that could have done the work for free. People walk differently. They give a short muscular push, then let the leg swing like a pendulum. One robotics result shows how much this matters. The Duke Humanoid adds a per-joint "torque activation" knob $\alpha\in[0,1]$ that can switch a motor fully off. A reinforcement-learning policy rewarded for low $\alpha$ discovers relaxation windows that cut the real robot's cost of transport (energy per unit weight per unit distance) from 1.13 to 0.77. Separately, a large body of work represents motion as discrete tokens, meaning short motion snippets chosen from a learned codebook, the way words are chosen from a vocabulary. These tokens can then be generated by language-model-style transformers (MotionGPT, Humanoid Locomotion as Next Token Prediction). None of these vocabularies can express "let go and let physics take over."

This note proposes a vocabulary in which each token is a short impulse-and-release segment. A token is a brief active command plus a committed decision about which joints go limp and for how long, together with the trajectory that the token predicts the limp joints will follow. A transformer picks tokens at a slow rate. A fast low-level loop executes them and re-engages the motors the moment the measured motion departs from the token's predicted passive path. The bet is that turning off torque is a committed, time-extended decision, not a per-timestep dial. Representing it as a discrete token with a duration and a predicted outcome should give the energy savings of passive walking without the two failure modes that dog it: chattering between on and off, and falling when pushed during a limp phase.

Core Claim

Continuous per-step relaxation control, as in the Duke policy, re-decides $\alpha$ at every control step. Nothing in it prevents rapid on/off switching or a timid hovering at intermediate $\alpha$, which acts as damping rather than true passivity. It also has no explicit expectation of what the passive limb should do, so a push during relaxation is caught only by whatever the policy happens to have learned. We claim that a codebook of impulse-and-release segments fixes both problems structurally. Each code $k$ decodes to three things:

  • a reference segment $q_k(t)$, the joint targets over the token's span;
  • a binary release schedule $a_k(t)\in\{0,1\}^J$ over the $J$ joints, saying which motors are on at each moment;
  • a predicted state trajectory $\hat{x}_k(t)$ for the segment.

These segments are learned from physically simulated rollouts, so every code corresponds to something the robot actually did under gravity and contact.

A fast guard then applies one rule:

$$\alpha_j(t) = a_{k,j}(t)\ \text{ unless }\ \lVert x(t)-\hat{x}_k(t)\rVert_W > \epsilon, \text{ in which case all joints switch to } \alpha=1.$$

Here $x(t)$ is the measured joint and body state, $W$ is a fixed weighting, and $\epsilon$ is a threshold. The rule runs at hundreds of hertz because it is a lookup and a comparison, not a transformer forward pass. This directly addresses the latency mismatch noted in the file: autoregressive token models run at about 40 Hz on Digit, while reflexes during unactuated phases need much faster loops.

What is new is the content of the token, not tokenization itself. Existing motion vocabularies quantize only kinematics or active motor commands (BEST, TM2T, MotionGPT, Humanoid Locomotion as Next Token Prediction). Existing passive-dynamics controllers use continuous per-step gating with no sequence structure (The Duke Humanoid).

Research Program

The work has four stages, built entirely on the open-source Duke Humanoid Isaac Gym environment and code (The Duke Humanoid). Its proximal actuators and low swing inertia make true passivity physically meaningful.

Stage 1: collect passive rollouts. We reproduce the Duke passive-reward policy and record rollouts across commanded speeds, turning rates and push disturbances. Each rollout logs joint states, targets, $\alpha$ and torques.

Stage 2: train the tokenizer. We train a vector-quantized autoencoder on fixed-length windows. This is an encoder whose output is snapped to the nearest of $K$ learned codebook vectors, as in the standard VQ setup reviewed in the file. The reconstruction target includes three parts: the joint-target segment, the release schedule binarized at a threshold, and the next-window state. Binarization is deliberate. It turns "release" into a categorical event, in the spirit of MeshTalk, whose categorical latents choose distinct events such as blinks instead of interpolated averages. We will sweep window length, because it sets how long a release is committed.

Stage 3: train the token planner. A causal transformer predicts the next token from the recent token and sensor history plus the velocity command, following the modality-aligned design of Humanoid Locomotion as Next Token Prediction. It is then fine-tuned with PPO in closed loop. The reward combines velocity tracking with measured electrical energy rather than a hand-weighted $\alpha$ penalty, so release is chosen because it saves energy, not because it is rewarded directly.

Stage 4: build and study the guard. We study the guard's threshold as a tunable trade-off between energy and robustness.

Secondary platform. We port the pipeline to a simulated Unitree G1 and evaluate it with the open-source GDAF tools from Biomechanical Comparisons Reveal Divergence of Human and Humanoid Gaits. That paper motivates the work. It shows that kinematically good imitation still yields knee-dominant power and distorted ankle work loops. It also shows a slow-speed collapse, where the comprehensive GDAF cost rises from 0.2941 at 1.5 m/s to 0.5106 at 0.5 m/s, which it attributes to erratic motor interventions. Committed release segments should reduce exactly that kind of intervention. The G1's geared actuators are less backdrivable, so we treat this platform as a test of whether the idea survives a less favorable body, not as the main result.

Baselines

Naive reference. A standard active PPO walking policy with a torque-squared penalty on the Duke simulation. The Duke paper reports its real-hardware analogue at a cost of transport of 1.13. This baseline shows how much any relaxation scheme buys.

Strongest existing method. The Duke passive policy itself, with continuous $\alpha$ and the passive-action reward. It reports up to 50% lower cost of transport in simulation at low speed, and 0.77 versus 1.13 on hardware with recovery from 300 N pushes. It is the only method in the file that demonstrably exploits passive dynamics on a humanoid. We run it from the released code with the same simulator, the same number of environment steps and parallel environments, and the same hyperparameter-search budget as our method. All energies are measured identically, as electrical cost of transport. Beating it means lower cost of transport at matched velocity-tracking error across the speed range, especially at 0.5 m/s. It also means a higher survival rate under pushes applied specifically during release phases.

Binary per-step gating. The same PPO setup, but with a categorical on/off actor for each joint's $\alpha$, implemented with the discretized categorical actor from RN-D. A variant adds a minimum dwell time, so a release must last several steps once chosen. This is the most dangerous competitor because it captures "discreteness" and "commitment" without any codebook.

Kinematic-only token planner. The token planner with $\alpha$ removed from the vocabulary, which is a faithful re-implementation of the next-token locomotion approach on the Duke body. It isolates whether putting release in the vocabulary matters.

Ablations of our method. The full method minus the guard isolates the predicted passive trajectory. The full method with a guard driven by a separately trained continuous dynamics model attached to the Duke policy tests whether the guard's value requires tokens at all. The full method with a continuous latent in place of the codebook, structured like the CVAE prior of Natural Humanoid Robot Locomotion with Generative Motion Prior, isolates quantization itself.

Why not just…?

Why not just keep the Duke continuous-$\alpha$ policy? It already works, and it is our primary baseline. Its weakness is structural rather than proven. Nothing in it forbids chattering or partial-$\alpha$ damping, and the file reports no measurement of switching frequency, of how often $\alpha$ sits at intermediate values, or of push recovery specifically during relaxation. We will measure all three on its released code. If it shows clean, long releases and robust recovery, our main argument weakens, and we will report that.

Why not just discretize $\alpha$ per step with a categorical actor and a minimum dwell time? This is a real threat. If dwell-time gating matches our cost of transport and push robustness, then commitment, not a learned vocabulary, is the active ingredient, and the paper's conclusion changes accordingly. We expect a gap for two reasons. A fixed dwell time cannot adapt release length to speed and gait phase, whereas codes learn variable durations. And gating alone supplies no predicted passive path for the guard. The paired ablation, in which the Duke policy is given a learned-model guard, separates these two contributions.

Why not just tokenize kinematics and let a stiff tracker execute them? Biomechanical Comparisons Reveal Divergence of Human and Humanoid Gaits shows that kinematic fidelity does not yield efficient kinetics. Kinematic tokens also carry no instruction to release, so the tracker will hold position, which is the stiffness Human-like compliant locomotion identifies as the root inefficiency. The kinematic-only token baseline tests this directly.

Why not just use a continuous diffusion or VAE latent? Continuous latents such as those in RoboGhost and the CVAE prior above cannot easily represent a hard on/off release. They also blur distinct segments into averages, and averaged release schedules would reintroduce intermediate damping. The continuous-latent ablation tests this. If it matches the codebook, quantization is unnecessary and we will say so.

Proposed Experiments

All headline experiments run in the Duke Isaac Gym environment over commanded speeds from slow to fast walking, with five seeds per method.

Metrics.

  • electrical cost of transport
  • velocity-tracking error
  • number of $\alpha$ switches per gait cycle and the fraction of time at intermediate $\alpha$
  • peak joint torque rate at token boundaries, which checks the quantization-induced torque spikes the file warns about
  • survival under 300 N pushes, split by whether the push lands during an active or a release phase

First decisive check. The full method must beat continuous-$\alpha$ Duke on cost of transport at 0.5 m/s without losing survival under release-phase pushes. If it does not, the core claim fails.

Second decisive check. The full method must also beat dwell-time binary gating. If it does not, the codebook adds nothing beyond commitment.

Third decisive check. Removing the guard should sharply reduce survival under release-phase pushes while leaving cost of transport roughly unchanged. That pattern would show the predicted passive trajectory is what makes aggressive release safe.

Guard threshold sweep. Sweeping $\epsilon$ traces an energy-versus-robustness curve, and we compare these curves across methods rather than single operating points.

Secondary G1 evaluation. We report the GDAF comprehensive cost and ankle torque-angle loop shape at 0.5 m/s and 1.5 m/s for our method and for a reproduced standard imitation policy. We use the reported 0.5106 and 0.2941 only as reference points, because our reproduction may differ. Success here means a smaller slow-speed degradation than the reproduced baseline. That would show committed releases reduce the erratic interventions Biomechanical Comparisons Reveal Divergence of Human and Humanoid Gaits blames for the slow-speed collapse.

Hardware. If access to the open Duke hardware is possible, we will repeat the cost-of-transport and push tests on the physical robot against the reported 0.77.