Introduction
Manipulating garments—unfolding a messy pile of laundry, folding a shirt, or hanging up a dress—remains a significant challenge in home robotics. Unlike rigid objects, clothing possesses nearly infinite degrees of freedom, frequently self-occludes, and undergoes drastic shape changes during handling. Most existing robotic solutions are task-specific, designed for a single action (like folding) on a specific garment geometry.
Researchers from Peking University and BIT have introduced UniGarmentManip, a unified framework that shifts away from task-specific policies. By teaching robots to understand the underlying topological structure shared by all garments within a category, a single model can handle multiple tasks and generalize to unseen clothing styles with minimal human guidance.
Methodology
The core innovation of UniGarmentManip is the use of self-supervised dense visual correspondence to map the "identity" of points across different garment shapes and deformations. The framework operates in three primary stages:
1. Learning Topological Correspondence
The learning process is decoupled into two distinct components:
- Cross-Deformation: Using "self-play" in simulation, the model learns to identify that a point on a flat shirt remains the same point when crumpled. This is trained using an InfoNCE loss, a contrastive learning technique that aligns matching points in feature space while separating non-matching ones.
- Cross-Object: To bridge different garments (e.g., from a T-shirt to a jacket), the system employs a Skeleton Merger. This extracts a sparse graph of keypoints representing the garment's topology. By interpolating between these points, the model creates a dense map that aligns different garments within the same category.
2. Coarse-to-Fine (C2F) Refinement
Initial correspondences can be imprecise, particularly near edges. The authors introduce an online refinement stage that identifies "failure points"—regions where the model is confident but geographically inaccurate in the canonical (flat) state—and applies additional training weight to those areas.
3. Few-Shot Functional Adaptation
Because a topological point (like a sleeve) is not always the optimal functional point for a specific task (like where to grab for folding), the model undergoes few-shot adaptation. With only 1–5 human demonstrations, the model fine-tunes its dense representations to prioritize the specific points required for tasks like hanging or folding.
Experimental Results
The framework was evaluated using the CLOTH3D dataset and real-world physical tests involving Franka Panda arms. UniGarmentManip was compared against state-of-the-art baselines such as FlingBot and ClothFunnels.
- Unfolding: The system achieved success rates between 83.6% and 86.9% across categories, outperforming FlingBot, which struggled with diverse garment scales.
- Folding: In the "Fold-FLING" task (starting from a messy state), UniGarmentManip achieved an ~82% success rate, significantly higher than ClothFunnels (~62%), which typically requires garments to be pre-flattened.
- Real-World Execution: In physical trials, the robot successfully folded five different real-world tops with an 11/15 success rate, demonstrating robust sim-to-real generalization.
Conclusion
UniGarmentManip demonstrates that leveraging topological consistency is a powerful strategy for deformable object manipulation. By decoupling identity from state and using skeletons as a topological bridge, the framework allows robots to reuse manipulation logic across entire categories of clothing. While challenges remain regarding extreme occlusions and sim-to-real gaps for complex textures, this work represents a significant step toward general-purpose laundry robotics.
Summary updated