Abstract
We propose the Dexterous Manipulation Graph (DMG) as a tool to address in-hand manipulation and reposition an object inside a robot's end-effector. This graph is used to plan a sequence of manipulation primitives so to bring the object to the desired end pose. This sequence of primitives is translated into motions of the robot to move the object held by the end-effector. We use a dual arm robot with parallel grippers to test our method on a real system and show successful planning and execution of in-hand manipulation.
Introduction
In robotics, "in-hand manipulation"—the ability to move an object once it has already been grasped—remains a notorious challenge. While humans achieve this effortlessly using intrinsic dexterity (multiple fingers and high tactile sensitivity), most industrial and research robots are equipped with simple two-fingered parallel grippers.
To move an object within such a gripper, traditional methods often require high-fidelity mathematical models of friction and dynamics, which are difficult to obtain in real-world scenarios. Alternatively, robots often resort to "regrasping"—dropping and picking the object back up—which is inherently slow and increases the risk of failure. This research introduces the Dexterous Manipulation Graph (DMG), a planning tool that leverages extrinsic dexterity. By using a dual-arm system where one arm holds the object and the other acts as an external "finger" to nudge it, robots can achieve complex repositioning without needing complex hands or perfect physics simulations.
The Dexterous Manipulation Graph (DMG) Framework
The core innovation of this work is the DMG, a geometric representation of the object's surface that guides manipulation planning. The construction of the DMG follows a systematic pipeline:
- Object Segmentation: The object’s 3D point cloud is divided into small areas called "supervoxels." The center of each area serves as a potential contact point for the pushing arm.
- Graph Construction: An initial graph connects adjacent supervoxels. This graph is refined by removing connections that would force a finger to move across sharp edges, ensuring smooth transitions.
- Orientation Discretization: For every contact point, the system calculates "admissible" finger orientations—those that allow the finger to contact the surface without colliding with the object's body.
- Final DMG Assembly: The final graph consists of nodes representing a unique (position, orientation) pair. An edge exists between nodes if the robot can move between them using a defined manipulation primitive.
Manipulation Primitives
The DMG plans paths by sequencing two primary manipulation primitives: Translation (sliding the contact point while keeping the object's orientation constant) and Rotation (keeping the contact point fixed while rotating the object). While the framework can theoretically support Rototranslation (combining both motions), the current implementation treats them as separate steps to manage computational complexity.
Using Dijkstra’s algorithm, the robot finds the shortest path in the DMG from the current grasp to the goal pose. This path is then translated into precise velocities for the dual-arm system using the Extended Cooperative Task Space (ECTS) framework.
Experimental Validation and Insights
The DMG was evaluated using several 3D mesh models, including simple cubes and complex, non-convex objects. Physical experiments were conducted on a Baxter robot, which is known for its compliant, "springy" joints. Despite these mechanical challenges, the system achieved repositioning accuracy within 10mm and 5 degrees of the target.
Key findings from the experiments include:
- Geometry over Dynamics: By focusing on surface connectivity and collision-free orientations rather than precise friction coefficients, the DMG proves robust in real-world settings where dynamic models often fail.
- Path Efficiency: The DMG successfully planned paths for non-convex objects where a simple straight-line movement would have caused collisions between the gripper and the object.
- Coordination Modes: The researchers found that an asymmetrical coordination mode—keeping the "holding" arm fixed while the "pushing" arm moved—provided higher precision than moving both arms simultaneously, largely due to the mechanical play in the robot's joints.
Conclusion and Future Work
The Dexterous Manipulation Graph provides a viable path for simple grippers to perform complex tasks. Future work aims to integrate "regrasping nodes" to allow the robot to transition between different connected components of the graph that are currently unreachable. Additionally, the authors plan to explore more fluid rototranslation movements and incorporate robust visual or tactile feedback to replace the current reliance on AprilTags for object tracking. This framework could eventually be extended to multi-fingered hands by coordinating multiple simultaneous paths on the graph.
Summary updated