Cooperative Mobile Manipulation using Learned Dynamical Systems

Saray Bakker, Wendelin Böhmer, Javier Alonso-MoraarXiv 2026 · arXiv

TL;DR

CALDS (Cooperative Adaptation of Learned Dynamical Systems) is a new framework that allows multiple mobile manipulators to collaborate using only single-agent skills learned from a few demonstrations. By intelligently selecting compatible policies and adjusting the timing—rather than the shape—of robot motions, it ensures collision-free coordination while strictly adhering to task constraints and demonstrated behaviors.

What Problem Does This Paper Address?

Cooperative mobile manipulation is a "holy grail" for domestic and industrial robotics, yet it faces a significant data bottleneck. Collecting high-quality demonstrations for multi-robot tasks is exponentially more difficult than for single robots. Furthermore, when robots operate in shared spaces, standard collision avoidance algorithms often force them to deviate significantly from their learned behaviors. These spatial deviations can lead to task failure or push the robot into "out-of-distribution" states where its learned policy becomes unreliable.

The authors address the challenge of how to enable seamless multi-robot collaboration using only a small library of single-agent skills, ensuring that robots don't just avoid hitting each other, but actually coordinate their timing to satisfy complex task dependencies (e.g., one robot waiting for another to finish cleaning a surface before placing an object on it).

Methods

The CALDS framework consists of two main centralized stages built upon Learned Dynamical Systems (DSs), which represent motion as a vector field that attracts the robot toward a goal.

  1. Cooperative Policy Selection: Instead of just picking the "best" individual policy for each robot, this layer evaluates combinations of policies. It ranks pairs of strategies based on how close they are to the original demonstrations and a simulated cost that accounts for potential collisions and task ordering.
  2. Temporal Adaptation: This is the core innovation. Rather than changing the spatial path of the robot (which might break the task), CALDS introduces a velocity scaling factor ($\alpha$). By optimizing this factor over a planning horizon, the system can "slow down" or "speed up" robots to avoid collisions and satisfy timing constraints. Because $\alpha$ only affects speed, the robot stays exactly on the spatial path prescribed by the learned DS.

The framework leverages JAX and FRAX for differentiable kinematics, allowing the system to solve these complex optimization problems using gradient-based methods (like L-BFGS) very efficiently.

The CALDS pipeline: (1) Learning single-agent skills, (2) Selecting the best policy combination, and (3) Optimizing timing via Temporal Adaptation.

Datasets

The authors evaluated CALDS using custom-generated scenarios in a MuJoCo-based simulation environment. They collected a small set of 5 teleoperated demonstrations per task-option to train the underlying Dynamical Systems (using CDSM and PUMA methods).

The framework was tested across three distinct benchmarks:

  • Restaurant: Robot 1 cleans a table while Robot 2 waits to place a cup on the cleaned area.
  • Dressing: Robot 1 assists a human in putting on a sleeve while Robot 2 positions a mirror.
  • Drawer: Robot 1 opens a drawer while Robot 2 places an item inside.
The three evaluation scenarios: Drawer opening, Dressing assistance, and Restaurant cleaning/serving.

Key Insights and Ideas

  • Timing over Geometry: The most powerful insight is that many multi-robot conflicts can be solved by adjusting when a robot reaches a point rather than where it goes. This preserves the "style" and convergence guarantees of the original learned behavior.
  • Policy Compatibility Matters: A policy that is individually optimal might be a nightmare to coordinate. By ranking combinations, CALDS can fall back to a "second-best" individual policy that allows for much smoother team coordination.
  • Stability Preservation: Because the temporal scaling factor $\alpha$ is kept strictly positive, the mathematical stability and goal-convergence properties of the original Dynamical Systems remain intact.

Key Results

CALDS was compared against several baselines, including standard Control Barrier Functions (CBFs) and ONCol-DMP.

  • Success Rates: CALDS achieved 91–100% success rates across all scenarios. In contrast, CBF-based methods struggled significantly in the Restaurant scenario (dropping as low as 2%) because they could not enforce the required task ordering (waiting for the table to be cleaned).
  • Path Fidelity: CALDS maintained significantly lower "Path Deviation" compared to spatial avoidance baselines. It stayed true to the demonstrated motions while others were forced to take wide, unintuitive detours to avoid collisions.
  • Efficiency: The system is computationally efficient, with planning times between 0.76s and 1.49s and action query times of just 0.18ms, making it viable for near real-time applications.
Quantitative results showing that CALDS (blue/cyan) maintains high success rates and low path deviation compared to baselines.

Limitations and Future Work

  • Centralization: The current selection and adaptation layers are centralized. Moving toward a decentralized architecture would improve scalability for larger robot swarms.
  • Dynamic Environments: While the system handles inter-robot coordination well, adapting to sudden, unpredictable changes in the environment (like a human walking through the workspace) at very high frequencies (100Hz+) remains a challenge.
  • Future Directions: The authors suggest integrating the framework into a single neural network conditioned on discrete behavioral markers and extending the work to direct human-robot collaboration, where uncertainty is much higher.

Summary updated

Abstract

Cooperative mobile manipulation requires robots to coordinate their motions while ensuring whole-body collision avoidance and preserving task-relevant behaviors. We propose CALDS (Cooperative Adaptation of Learned Dynamical Systems), a framework that enables mobile manipulators to adapt learned single-agent behaviors for coordinated execution. Building on individual dynamical systems (DSs) learned from demonstrations, CALDS enables cooperative adaptation through two key components: Cooperative Policy Selection, which identifies compatible combinations of learned policies, and Temporal Adaptation, which ensures collision-free cooperation while preserving the spatial structure and convergence properties of the original behaviors. We evaluate CALDS across three diverse scenarios: assistive dressing, restaurant cleaning and serving, and drawer opening and object placement. Quantitative results demonstrate its effectiveness in improving task success and enabling safe, coordinated simultaneous execution while maintaining fidelity to demonstrated behaviors.
BibTeX
@misc{bakker2026cooperative,
  author = {Saray Bakker and Wendelin Böhmer and Javier Alonso-Mora},
  title = {{Cooperative Mobile Manipulation using Learned Dynamical Systems}},
  year = {2026},
  eprint = {2611.21198},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.21198},
}
Figure