TL;DR
Marope is a hierarchical multi-agent reinforcement learning (MARL) framework that enables humanoid robots to cooperatively perform long rope skipping. By combining decentralized low-level rope manipulation with a centralized high-level scheduler and a diversity-driven training regime, the system achieves robust coordination between two robot turners and a jumping player (human or robot) in both simulation and real-world environments.
What Problem Does This Paper Address?
While humanoid robots have mastered individual athletic skills like running, dancing, and parkour, tasks requiring tight coordination between multiple agents and interaction with complex objects remain a significant challenge. Long rope skipping is a prime example: it requires two turners to generate a coherent rope motion through whole-body control while synchronizing perfectly with a player’s jumping rhythm.
The task is difficult because the rope is a deformable, underactuated object whose intermediate configuration cannot be directly controlled. Furthermore, the system must adapt to diverse jumping styles in "open-world" scenarios, where a human player's rhythm might be unknown or inconsistent.
Methods
The authors propose Marope, a hierarchical framework designed to decompose the complexity of the task:
- Low-Level Decentralized Manipulation: Two humanoid turners learn decentralized policies using MAPPO (Multi-Agent Proximal Policy Optimization). Instead of tracking specific rope trajectories, they use a compact command space (specifying rotation center, width, and angular velocity). This allows the robots to maintain balance and even move (translate or turn) while swinging the rope.
- High-Level Centralized Scheduling: A scheduling policy ($\pi_{sched}$) dynamically adjusts the low-level commands to synchronize the rope's phase with the player's jumping rhythm, minimizing the risk of collisions.
- Diverse Player Discovery: To ensure the scheduler can handle different people, the authors use an Integral Probability Metric (IPM)-based diversity objective. This discovers a wide variety of jumping behaviors during training, which serves as data augmentation to improve the scheduler's generalization.
Datasets
The researchers evaluated Marope using:
- Simulation: Massive parallel simulations were conducted in Isaac Lab, using a lumped multi-body system to approximate rope physics.
- Real-World Deployment: The system was deployed on Unitree G1 humanoid robots. An optical motion capture (MoCap) system provided real-time positions for the rope markers, robot base links, and the jumping player.
Key Insights and Ideas
- Abstraction over Trajectory: By defining the rope motion through a "reference center" and "rotation axis" rather than explicit point-by-point trajectories, the robots gain the mobility to adjust their stance and position dynamically.
- Symmetry as a Prior: The observation space is constructed symmetrically for both turners, allowing them to share the same policy parameters, which significantly improves training efficiency.
- Training for the "Unknown": The "aha moment" here is that training against a single, perfect jumping policy leads to brittle coordination. By explicitly optimizing for diverse jumping styles (visualized via PCA in the paper), the robots learn a much more robust scheduling policy that can adapt to the "messy" rhythms of human jumpers.
Key Results
Marope demonstrated superior performance across several metrics compared to single-agent and rule-based baselines:
- Stability and Accuracy: In simulation, Marope achieved a rotation tracking error of 1.719, significantly lower than the single-agent baseline (2.733). It also showed the lowest "feet slippage," indicating better whole-body balance.
- Coordination Success: The system achieved a 78.7% jump completion rate, compared to only 35.8% for a non-adaptive rule-based scheduler.
- Real-World Robustness: The robots successfully coordinated with both other humanoid robots and human players, despite the large differences in body morphology and jumping styles.
Limitations and Future Work
The authors acknowledge that the current framework is specifically tailored to long rope skipping and may not generalize immediately to other cooperative tasks. Currently, the system only supports a single jumper and relies on an external MoCap system for high-fidelity sensing.
Future work aims to extend the framework to more complex scenarios like "Double Dutch" (two ropes) and multiple jumpers. A major goal is to move away from MoCap toward purely on-board vision and proprioceptive sensors, enabling these robots to skip rope "in the wild" with human partners anywhere.
Summary updated