Cooperative Long Rope Skipping via Multi-Agent Reinforcement Learning

Zihao Wang, Shijie Peng, Kerui Wu, Yu Huang, Ruiqi Xue, Dong Liu, Tian Xu, Lei Yuan, Yang YuarXiv 2026 · arXiv

TL;DR

Marope is a hierarchical multi-agent reinforcement learning (MARL) framework that enables humanoid robots to cooperatively perform long rope skipping. By combining decentralized low-level rope manipulation with a centralized high-level scheduler and a diversity-driven training regime, the system achieves robust coordination between two robot turners and a jumping player (human or robot) in both simulation and real-world environments.

What Problem Does This Paper Address?

While humanoid robots have mastered individual athletic skills like running, dancing, and parkour, tasks requiring tight coordination between multiple agents and interaction with complex objects remain a significant challenge. Long rope skipping is a prime example: it requires two turners to generate a coherent rope motion through whole-body control while synchronizing perfectly with a player’s jumping rhythm.

The task is difficult because the rope is a deformable, underactuated object whose intermediate configuration cannot be directly controlled. Furthermore, the system must adapt to diverse jumping styles in "open-world" scenarios, where a human player's rhythm might be unknown or inconsistent.

Methods

The authors propose Marope, a hierarchical framework designed to decompose the complexity of the task:

  1. Low-Level Decentralized Manipulation: Two humanoid turners learn decentralized policies using MAPPO (Multi-Agent Proximal Policy Optimization). Instead of tracking specific rope trajectories, they use a compact command space (specifying rotation center, width, and angular velocity). This allows the robots to maintain balance and even move (translate or turn) while swinging the rope.
  2. High-Level Centralized Scheduling: A scheduling policy ($\pi_{sched}$) dynamically adjusts the low-level commands to synchronize the rope's phase with the player's jumping rhythm, minimizing the risk of collisions.
  3. Diverse Player Discovery: To ensure the scheduler can handle different people, the authors use an Integral Probability Metric (IPM)-based diversity objective. This discovers a wide variety of jumping behaviors during training, which serves as data augmentation to improve the scheduler's generalization.
Overview of the Marope framework, showing the hierarchical coordination and diversity discovery modules.

Datasets

The researchers evaluated Marope using:

  • Simulation: Massive parallel simulations were conducted in Isaac Lab, using a lumped multi-body system to approximate rope physics.
  • Real-World Deployment: The system was deployed on Unitree G1 humanoid robots. An optical motion capture (MoCap) system provided real-time positions for the rope markers, robot base links, and the jumping player.

Key Insights and Ideas

  • Abstraction over Trajectory: By defining the rope motion through a "reference center" and "rotation axis" rather than explicit point-by-point trajectories, the robots gain the mobility to adjust their stance and position dynamically.
  • Symmetry as a Prior: The observation space is constructed symmetrically for both turners, allowing them to share the same policy parameters, which significantly improves training efficiency.
  • Training for the "Unknown": The "aha moment" here is that training against a single, perfect jumping policy leads to brittle coordination. By explicitly optimizing for diverse jumping styles (visualized via PCA in the paper), the robots learn a much more robust scheduling policy that can adapt to the "messy" rhythms of human jumpers.
Visualization of discovered jumping behaviors and phase tracking performance.

Key Results

Marope demonstrated superior performance across several metrics compared to single-agent and rule-based baselines:

  • Stability and Accuracy: In simulation, Marope achieved a rotation tracking error of 1.719, significantly lower than the single-agent baseline (2.733). It also showed the lowest "feet slippage," indicating better whole-body balance.
  • Coordination Success: The system achieved a 78.7% jump completion rate, compared to only 35.8% for a non-adaptive rule-based scheduler.
  • Real-World Robustness: The robots successfully coordinated with both other humanoid robots and human players, despite the large differences in body morphology and jumping styles.
Real-world deployment scenarios including humanoid-humanoid and humanoid-human coordination.

Limitations and Future Work

The authors acknowledge that the current framework is specifically tailored to long rope skipping and may not generalize immediately to other cooperative tasks. Currently, the system only supports a single jumper and relies on an external MoCap system for high-fidelity sensing.

Future work aims to extend the framework to more complex scenarios like "Double Dutch" (two ropes) and multiple jumpers. A major goal is to move away from MoCap toward purely on-board vision and proprioceptive sensors, enabling these robots to skip rope "in the wild" with human partners anywhere.

Summary updated

Abstract

Humans exhibit remarkable motor agility, enabling a wide range of dynamic skills such as running and jumping, which highlights the great potential of humanoid robots for athletic locomotion. Among athletic sports, long rope skipping requires two rope turners to cooperatively swing the rope while adapting to a player under different jumping rhythms, making it a meaningful yet challenging task for humanoid robots. Although existing methods for humanoid sports have achieved success in single-agent and interaction-free settings, such as running, dancing, and parkour, task scenarios that require precise coordination among multiple participants remain largely unexplored. To this end, we propose Marope, a multi-agent reinforcement learning (MARL) framework for cooperative long rope skipping with multiple humanoid robots. Specifically, Marope adopts a hierarchical reinforcement learning framework for policy training. At the lower level, it learns decentralized rope manipulation policies through MARL, while at the upper level, a centralized scheduling policy is trained to coordinate the execution of the lower-level policies. To improve generalization across different player behavioral styles, Marope further incorporates diverse jumping policies into cooperative game training. We evaluate our approach on Unitree G1 humanoid robots in both simulation and real-world settings. Experimental results demonstrate that Marope outperforms various baselines, achieving more efficient and stable rope manipulation as well as more robust and adaptable cooperation with varied players.
BibTeX
@misc{wang2026cooperative,
  author = {Zihao Wang and Shijie Peng and Kerui Wu and Yu Huang and Ruiqi Xue and Dong Liu and Tian Xu and Lei Yuan and Yang Yu},
  title = {{Cooperative Long Rope Skipping via Multi-Agent Reinforcement Learning}},
  year = {2026},
  eprint = {2606.08064},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2606.08064},
}
Figure