TL;DR
CALDS (Cooperative Adaptation of Learned Dynamical Systems) is a new framework that allows multiple mobile manipulators to collaborate using only single-agent skills learned from a few demonstrations. By intelligently selecting compatible policies and adjusting the timing—rather than the shape—of robot motions, it ensures collision-free coordination while strictly adhering to task constraints and demonstrated behaviors.
What Problem Does This Paper Address?
Cooperative mobile manipulation is a "holy grail" for domestic and industrial robotics, yet it faces a significant data bottleneck. Collecting high-quality demonstrations for multi-robot tasks is exponentially more difficult than for single robots. Furthermore, when robots operate in shared spaces, standard collision avoidance algorithms often force them to deviate significantly from their learned behaviors. These spatial deviations can lead to task failure or push the robot into "out-of-distribution" states where its learned policy becomes unreliable.
The authors address the challenge of how to enable seamless multi-robot collaboration using only a small library of single-agent skills, ensuring that robots don't just avoid hitting each other, but actually coordinate their timing to satisfy complex task dependencies (e.g., one robot waiting for another to finish cleaning a surface before placing an object on it).
Methods
The CALDS framework consists of two main centralized stages built upon Learned Dynamical Systems (DSs), which represent motion as a vector field that attracts the robot toward a goal.
- Cooperative Policy Selection: Instead of just picking the "best" individual policy for each robot, this layer evaluates combinations of policies. It ranks pairs of strategies based on how close they are to the original demonstrations and a simulated cost that accounts for potential collisions and task ordering.
- Temporal Adaptation: This is the core innovation. Rather than changing the spatial path of the robot (which might break the task), CALDS introduces a velocity scaling factor ($\alpha$). By optimizing this factor over a planning horizon, the system can "slow down" or "speed up" robots to avoid collisions and satisfy timing constraints. Because $\alpha$ only affects speed, the robot stays exactly on the spatial path prescribed by the learned DS.
The framework leverages JAX and FRAX for differentiable kinematics, allowing the system to solve these complex optimization problems using gradient-based methods (like L-BFGS) very efficiently.
Datasets
The authors evaluated CALDS using custom-generated scenarios in a MuJoCo-based simulation environment. They collected a small set of 5 teleoperated demonstrations per task-option to train the underlying Dynamical Systems (using CDSM and PUMA methods).
The framework was tested across three distinct benchmarks:
- Restaurant: Robot 1 cleans a table while Robot 2 waits to place a cup on the cleaned area.
- Dressing: Robot 1 assists a human in putting on a sleeve while Robot 2 positions a mirror.
- Drawer: Robot 1 opens a drawer while Robot 2 places an item inside.
Key Insights and Ideas
- Timing over Geometry: The most powerful insight is that many multi-robot conflicts can be solved by adjusting when a robot reaches a point rather than where it goes. This preserves the "style" and convergence guarantees of the original learned behavior.
- Policy Compatibility Matters: A policy that is individually optimal might be a nightmare to coordinate. By ranking combinations, CALDS can fall back to a "second-best" individual policy that allows for much smoother team coordination.
- Stability Preservation: Because the temporal scaling factor $\alpha$ is kept strictly positive, the mathematical stability and goal-convergence properties of the original Dynamical Systems remain intact.
Key Results
CALDS was compared against several baselines, including standard Control Barrier Functions (CBFs) and ONCol-DMP.
- Success Rates: CALDS achieved 91–100% success rates across all scenarios. In contrast, CBF-based methods struggled significantly in the Restaurant scenario (dropping as low as 2%) because they could not enforce the required task ordering (waiting for the table to be cleaned).
- Path Fidelity: CALDS maintained significantly lower "Path Deviation" compared to spatial avoidance baselines. It stayed true to the demonstrated motions while others were forced to take wide, unintuitive detours to avoid collisions.
- Efficiency: The system is computationally efficient, with planning times between 0.76s and 1.49s and action query times of just 0.18ms, making it viable for near real-time applications.
Limitations and Future Work
- Centralization: The current selection and adaptation layers are centralized. Moving toward a decentralized architecture would improve scalability for larger robot swarms.
- Dynamic Environments: While the system handles inter-robot coordination well, adapting to sudden, unpredictable changes in the environment (like a human walking through the workspace) at very high frequencies (100Hz+) remains a challenge.
- Future Directions: The authors suggest integrating the framework into a single neural network conditioned on discrete behavioral markers and extending the work to direct human-robot collaboration, where uncertainty is much higher.
Summary updated