Abstract
We present ECHO, an edge-cloud framework for language-driven whole-body control of humanoid robots. A cloud-hosted diffusion-based text-to-motion generator synthesizes motion references from natural language instructions, while an edge-deployed reinforcement-learning tracker executes them in closed loop on the robot. The two modules are bridged by a compact, robot-native 38-dimensional motion representation that encodes joint angles, root planar velocity, root height, and a continuous 6D root orientation per frame, eliminating inference-time retargeting from human body models and remaining directly compatible with low-level PD control. Real-world experiments on a Unitree G1 humanoid demonstrate stable execution of diverse text commands with zero hardware fine-tuning.
Introduction
Bridging the gap between high-level human language and low-level robot motors is notoriously difficult. Current approaches usually fall into three traps:
- End-to-End Bottlenecks: Running large language-action models directly on a robot's limited on-board computer causes high latency, which is dangerous for a balancing humanoid.
- The "Retargeting" Tax: Most systems generate motion for human body models (like SMPL) and then try to "map" it to the robot's specific skeleton in real-time. This is computationally expensive and often leads to physical inconsistencies like "foot sliding" or loss of balance.
- Modularity vs. Portability: Many "retargeting-free" methods use "black-box" latent spaces that are hard to debug or move to a different robot model without retraining everything from scratch.
ECHO solves this by introducing a robot-native interface that allows the cloud to speak the robot's "language" directly, ensuring the on-board controller only has to focus on physics and stability.
Framework Architecture
The framework is split into two distinct parts, bridged by a 38-dimensional motion representation:
- The Cloud (The "Dreamer"): A diffusion-based text-to-motion generator. It uses a 1D convolutional UNet conditioned on CLIP text embeddings. Unlike models that output human poses, this model outputs a 38-dimensional "robot-native" representation (29 joint angles, root velocity, height, and 6D orientation). It uses DDIM sampling to generate a full motion sequence in just 10 steps (~1 second).
- The Edge (The "Doer"): A reinforcement learning (RL) tracker deployed on the robot. It uses a Teacher-Student paradigm. A "Teacher" policy learns to follow motions in simulation with perfect information. This is then distilled into a "Student" policy that only uses the robot's actual sensors (proprioception).
Training and Evaluation
The authors retargeted the HumanML3D dataset to the Unitree G1 robot skeleton offline, creating a massive library of robot-native motions paired with text descriptions. To evaluate performance, the paper introduces two robot-centric benchmarks:
- Motion Safety Score (MSS): Quantifies how well the generated motion respects the robot's physical limits (torque, joint range).
- Root Trajectory Consistency (RTC): Measures how closely the robot's actual path matches the "intended" path from the text command.
Key Results and Validation
ECHO achieved an FID (Fréchet Inception Distance) of 0.029, a 21.6% improvement over previous best-in-class models, meaning its generated motions are significantly more realistic and diverse. In real-world tests on a physical Unitree G1 humanoid, the system achieved a 100% success rate across 80 trials of diverse commands (punching, waving, kicking, squatting). Crucially, the robot was able to perform these tasks in the real world immediately after simulation training, with no hardware-specific fine-tuning required.
Conclusion
ECHO demonstrates that decoupling semantic understanding from physical execution allows humanoids to handle complex instructions without sacrificing the high-frequency control loops needed for balance. While currently "blind" to external obstacles, the authors suggest that the next step is evolving ECHO into a Vision-Language-Action (VLA) architecture. This would involve feeding real-time visual feedback back into the loop so the robot can perform agile control in cluttered, unpredictable environments.
Summary updated