Abstract
Recent advancements in robotic manipulation through reinforcement learning (RL) and imitation learning (IL) have demonstrated potential in tasks such as opening drawers or cupboards. However, these techniques often fail to generalize to unseen objects, largely due to the high-dimensional action space inherent in joint-level control. This paper proposes an alternative approach that decouples the task of learning "what to do" (skill dynamics) from "how to do it" (whole-body control). We formulate the RL problem as determining the skill dynamics for a disembodied virtual manipulator. Subsequently, a whole-body robotic controller is optimized to execute these motions by solving a quadratic programming (QP) model that accounts for robotic singularity and kinematic constraints. Our experiments on complex articulated objects demonstrate that this decoupled approach significantly improves generalization to unseen objects with large intra-class variations, outperforming traditional end-to-end RL and IL baselines in both success rates and motion compliance.
Introduction
Teaching high-degree-of-freedom (DoF) robots to interact with articulated objects—such as cabinets, drawers, and doors—remains a fundamental challenge in robotics. Standard end-to-end approaches typically struggle with two main issues. First, the vast search space of a 13-DoF robot makes exploration in RL computationally expensive and often unproductive. Second, imitation learning is highly sensitive to distribution shifts; a robot trained on one specific cabinet often fails when presented with a model of a different height or handle orientation.
The core of this difficulty lies in the tight coupling between the task-specific skill and the robot's physical configuration. To address this, we propose a framework that separates high-level manipulation skills from low-level kinematic execution.
Methodology
Stage 1: Learning Skill Dynamics (The "What")
Instead of training the full robot body from the outset, we train a disembodied virtual manipulator (a floating 6-DoF gripper) using Soft Actor-Critic (SAC). By focusing solely on the physics of the interaction—how the gripper must approach, grasp, and pull a handle—the RL agent operates in a significantly reduced action space. This abstraction allows the agent to learn the fundamental "skill dynamics" of the task without being bogged down by the complexities of whole-body joint coordination.
Stage 2: Whole-Body Control (The "How")
To translate the virtual gripper's trajectory into real-world robot motion, we utilize a Quadratic Programming (QP) solver. This mathematical bridge maps the 6-DoF task-space goals to the robot's high-dimensional joint space (e.g., a Franka arm on a mobile base). The QP model optimizes for joint velocities while respecting critical physical constraints, such as:
- Singularity Avoidance: Preventing the robot from entering mathematical "dead zones" where joints lock up.
- Kinematic Limits: Ensuring the arm and base stay within safe operational boundaries.
- Smoothness: Minimizing unnecessary base movement and ensuring compliant, human-like motion.
Perception and State Representation
The framework utilizes PointNet ensembles to process 3D point clouds from onboard cameras. The system distinguishes between "static" states (object geometry and size) and "dynamic" states (the current position of the articulated part), providing the RL agent with a clean, structured representation of the environment.
Experimental Evaluation
Setup
We evaluated our framework using the ManiSkill benchmark, focusing on the task of opening cabinet drawers. The dataset included 25 verified cabinet models, with 15 used for training and 10 reserved for testing to evaluate category-level generalization.
Results
Our decoupled approach demonstrated superior performance across both training and testing sets:
- Success Rates: The proposed method achieved a 74% success rate on training cabinets and maintained a 51% success rate on unseen test cabinets.
- Baseline Comparison: Traditional end-to-end SAC and Behavior Cloning (BC) baselines struggled significantly with generalization, with success rates on the test set dropping to between 10% and 13%.
- Motion Quality: Qualitative analysis showed that the QP-based controller produced smoother, more reliable trajectories. In contrast, end-to-end RL agents often exhibited "jerky" motions or unnatural joint configurations when approaching object boundaries.
Discussion and Conclusion
The success of this framework highlights the value of abstraction in robotic learning. By stripping away the robot's body during the skill-acquisition phase, the agent focuses entirely on the task's dynamics, leading to more robust and transferable skills. The use of a QP solver provides a reliable, constraint-aware bridge to physical execution that end-to-end learning currently struggles to replicate.
Future work will focus on extending this framework to multi-stage tasks and addressing the sim-to-real gap, specifically regarding sensor noise and friction modeling in physical environments. By decoupling the "mind" of the task from the "body" of the robot, we move closer to general-purpose robotic manipulators capable of navigating the diversity of the real world.
Summary updated