TL;DR
Researchers at USC have developed ARMFUL, a pipeline that enables a single robot arm to pick up large, heavy objects like chairs and boxes by wrapping its entire structure around them. They introduced WAG, the first large-scale dataset for whole-arm grasping with 1.7 million configurations, and used a diffusion model to translate object point clouds into successful "hugging" poses.
What Problem Does This Paper Address?
Standard robotic grasping is "end-effector centric." Whether it's a parallel-jaw gripper or a suction cup, the focus is almost entirely on the robot's "hand." However, if you try to pick up a large IKEA box or a chair with just your fingertips, you’ll likely fail—either the object is too wide for your span or too heavy for your grip.
In these cases, humans use Whole-Arm Manipulation (WAM): we wrap our arms around the object, using our forearms and chest for support. For robots, learning this is incredibly difficult because:
- High-Dimensionality: You aren't just planning a 6-DoF pose for a hand; you're coordinating every joint in the arm to avoid collisions while maximizing contact.
- Data Scarcity: Existing datasets focus on small objects and grippers. Teleoperating a robot arm to perform complex "wrapping" motions is tedious and doesn't scale.
Methods
The authors tackle this with a two-pronged approach: a massive synthetic dataset and a generative model pipeline.
1. WAG (Whole-Arm Grasp Dataset)
To generate data without human teleoperation, they used a "sample-and-test" strategy in a GPU-parallelized simulator (Isaac Lab).
- Sampling: They sampled millions of arm configurations directly in joint space.
- Filtering: They discarded any poses that collided with the table or the object.
- The "Squeeze and Lift" Test: Surviving poses were executed using scripted primitives. The arm "squeezes" (moving the elbow and wrist joints inward) and then "lifts."
- The Stress Test: To ensure robustness, they didn't just check if the object lifted. They "shook" the arm and applied external 30N forces. Only configurations that held onto the object through this "jittering" made it into the final 1.7M-entry dataset.
2. The ARMFUL Pipeline
At deployment, the robot sees a new object as a point cloud.
- Grasp Generation: A conditional diffusion model (DDPM) takes the point cloud (encoded via a DP3 encoder) and the arm's initial state to generate a candidate whole-arm grasp configuration ($q_{grasp}$).
- Motion Planning: Since the diffusion model only predicts the final pose, the team uses RRT-Connect to find a collision-free path from the start position to that "hug" pose.
- Execution: Once in position, the robot executes the same squeeze-and-lift primitives used during data generation.
Datasets
- WAG (Whole-Arm Grasp Dataset): The core contribution. It contains 1.7 million successful, stress-tested grasp configurations across 793 objects derived from the DG16M dataset.
- IKEA Object Set: 10 commercially available IKEA items (stools, shelves, bins) used for real-world validation. These were not seen during the training of the diffusion model.
Key Insights and Ideas
- Joint-Space vs. Task-Space: While most grasping research focuses on where the hand should be in 3D space, this paper argues that for whole-arm tasks, sampling directly in joint space is more effective for discovering valid "wraps" that respect the robot's kinematic limits.
- The Power of Perturbation: The authors found that training on "nominal" successful lifts wasn't enough. By adding a "stress test" (shaking and pushing) during dataset generation, the resulting model was significantly more successful on held-out objects.
- Diffusion for Multi-modality: There are many ways to hug a box (from the left, from the right, high, or low). Diffusion models are uniquely suited to capturing this multi-modal distribution, allowing the robot to choose a feasible wrap based on its starting position.
Key Results
- Simulation: ARMFUL achieved an 85.4% lift success rate on training objects and 69.9% on completely unseen (held-out) objects. It significantly outperformed a "middle-height wrap" heuristic and nearest-neighbor retrieval baselines.
- Real-World: On a physical Franka Panda arm, the system successfully lifted 10 different IKEA objects in 70 out of 100 trials.
- Robustness: In simulation, 91% of ARMFUL's successful lifts on held-out objects survived the perturbation stress test, compared to only 84% for the hand-designed heuristic.
Limitations and Future Work
- Sim-to-Real Perception Gap: The system relies on clean point clouds. In the real world, thin parts of furniture (like chair legs) often appear "noisy" or incomplete to depth cameras, leading to planning failures.
- Rigid Body Assumption: The simulator treats everything as rigid. In real-world tests, a fabric box deformed around the arm, and some IKEA furniture joints rotated—behaviors the model hadn't learned to account for.
- Hardcoded Primitives: Currently, the "squeeze" and "lift" motions are hand-scripted. The authors suggest that future work could involve learning the entire manipulation trajectory rather than just the static grasp configuration.
For more details, check out the project website.
Summary updated