CART-MPC: Coordinating Assistive Devices for Robot-Assisted Transferring with Multi-Agent Model Predictive Control

Ruolin Ye, Shuaixing Chen, Yunting Yan, Joyce Yang, Christina Ge, Jose A. Barreiros, Kate Tsui, Tom Silver, Tapomayukh Bhattacharjee

Introduction

Patient transfer—moving an individual from a bed to a wheelchair—is a physically taxing task for caregivers and a significant challenge for assistive robotics. Standard collaborative robots, such as the Kinova Gen3, typically have a payload capacity of around 5kg, which is insufficient for lifting a human. While mechanical aids like Hoyer slings are used by humans to manage this weight, automating the process is difficult. Specifically, fastening deformable sling straps to a swinging, unstable sling bar requires coordinating non-linear dynamics and generalizing across various hook designs.

The CART-MPC (Coordinating Assistive Devices for Robot-Assisted Transferring) framework addresses this by treating passive assistive devices as "intelligent agents." By combining multi-agent model predictive control (MPC) with a knot-theory-inspired cost function, the system enables robots to autonomously secure straps, overcoming traditional payload limitations.

Methodology

The CART-MPC system conceptualizes the robot arm, the wheelchair, and the Hoyer sling as a coordinated multi-agent team.

  1. Instrumented Hardware: A standard Hoyer sling was modified with actuators (electric wheels and a servo-rotated bar) and sensors to receive commands via ROS.
  2. Turn-Taking Multi-Agent MPC: The control algorithm employs a decentralized approach where the robot and the sling bar alternate actions. The robot positions the strap while the bar rotates to meet it or stabilize the hook, optimizing for a shared goal.
  3. Learned Dynamics: To manage the computational complexity of modeling deformable objects, a neural network (MLP) predicts strap movement using keypoint-based representations. These keypoints are detected via GroundedSAM2 and tracked using optical flow.
  4. Knot-Theory Cost Function: The system uses "linking numbers" from knot theory to mathematically define the success of a tie. An LSTM is trained to estimate this value directly from visual keypoints, enabling real-time optimization.
Simulation of the multi-agent coordination between the robot arm (strap) and the instrumented Hoyer sling bar (hook).

Key Insights and Ideas

A central innovation of this work is the shift from increasing robot strength to increasing the intelligence of existing tools. By actuating the sling bar, the task becomes a collaborative effort.

  • The "Wiggle" Factor: Actuating the sling bar allows for a "wiggling" motion that increases error tolerance. If the robot's strap placement is slightly inaccurate, the bar can adjust its position to catch the strap.
  • Continuous Progress in Discrete Tasks: While tying a strap is a binary outcome, the linking number provides a continuous gradient. This allows the MPC to optimize movement by recognizing when it is getting closer to a successful connection.
The 'Wiggle' and 'Stabilize' motions of the sling bar help the system compensate for strap positioning errors.

Evaluation and Results

The researchers evaluated CART-MPC primarily in the RCareWorld simulation environment, testing across five hook geometries, three sling materials, and five diverse patient models ("CareAvatars").

  • Performance: CART-MPC achieved a single-strap success rate of 0.82, significantly higher than single-agent baselines with fixed (0.54) or passive (0.59) bars.
  • Generalization: The system successfully generalized to novel hook shapes and patient body types not seen during training.
  • Sim-to-Real Validation: In physical tests with a Kinova Gen3 arm and a manikin, the system achieved a 62% success rate in zero-shot strap tying.
  • Full Task Completion: Simulation results confirmed the system could complete a full four-strap tie and safely lift a care recipient for at least five seconds.

Limitations and Future Work

Despite its success, the system faces challenges with "out-of-distribution" materials; for instance, very stiff straps behave differently than the compliant ones used in training. Additionally, the current 2D keypoint tracking can lead to errors where the strap appears close to the hook but is not actually engaged. Future improvements may involve incorporating 3D depth information and addressing the dynamics of active human recipients who may move during the transfer. Expanding the system's autonomy to include positioning the sling under the patient remains a key objective.

Summary updated

Abstract

Bed-to-wheelchair transferring is a ubiquitous activity of daily living (ADL), but especially challenging for caregiving robots with limited payloads. We develop a novel algorithm that leverages the presence of other assistive devices: a Hoyer sling and a wheelchair for coarse manipulation of heavy loads, alongside a robot arm for fine-grained manipulation of deformable objects (Hoyer sling straps). We instrument the Hoyer sling and wheelchair with actuators and sensors so that they can become intelligent agents in the algorithm. We then focus on one subtask of the transferring ADL -- tying Hoyer sling straps to the sling bar -- that exemplifies the challenges of transfer: multi-agent planning, deformable object manipulation, and generalization to varying hook shapes, sling materials, and care recipient bodies. To address these challenges, we propose CART-MPC, a novel algorithm based on turn-taking multi-agent model predictive control that uses a learned neural dynamics model for a keypoint-based representation of the deformable Hoyer sling strap, and a novel cost function that leverages linking numbers from knot theory and neural amortization to accelerate inference. We validate it in both RCareWorld simulation and real-world environments. In simulation, CART-MPC successfully generalizes across diverse hook designs, sling materials, and care recipient body shapes. In the real world, we show zero-shot sim-to-real generalization capabilities to tie deformable Hoyer sling straps on a sling bar towards transferring a manikin from a hospital bed to a wheelchair. See our website for supplementary materials: https://emprise.cs.cornell.edu/cart-mpc/.
BibTeX
@inproceedings{ye2025cart,
  author = {Ruolin Ye and Shuaixing Chen and Yunting Yan and Joyce Yang and Christina Ge and Jose A. Barreiros and Kate Tsui and Tom Silver and Tapomayukh Bhattacharjee},
  title = {{CART-MPC: Coordinating Assistive Devices for Robot-Assisted Transferring with Multi-Agent Model Predictive Control}},
  booktitle = {HRI},
  year = {2025},
  doi = {10.1109/hri61500.2025.10974069},
}
Figure