Abstract
Cloth unfolding and folding are fundamental tasks in autonomous robotic cloth manipulation, representing a significant frontier in Physical AI. Driven by recent advances in deep learning, this field has expanded rapidly. This systematic review identifies and summarizes progress in deep learning-based cloth manipulation, following PRISMA guidelines to analyze 41 relevant papers published between 2019 and 2024. While current methods show impressive performance, challenges remain regarding irregular cloth sizes, diverse initial states, and the "Sim2Real" gap. This paper proposes a new taxonomy of six learning and control paradigms and discusses the future of multi-modal sensing and standardized metrics in the field.
Introduction
Cloth manipulation is often considered a "grand challenge" for robotics. Unlike rigid objects, cloth possesses an infinite-dimensional configuration space, complex non-linear dynamics, and a tendency for constant self-occlusion. While deep learning (DL) has accelerated progress, the research landscape has remained fragmented. This review provides a structured analysis of task contents, datasets, and primitive actions to offer a roadmap for future research in Physical AI.
Methodology
To ensure a rigorous and reproducible analysis, the authors followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. The selection process involved filtering 655 records from major databases (IEEE Xplore, Scopus, Web of Science, and ACM Digital Library) down to 41 high-quality papers based on strict inclusion criteria, such as peer-review status and a focus on deep learning-based unfolding or folding.
The Six Paradigms of Robotic Cloth Manipulation
The review moves beyond simple "supervised vs. reinforcement learning" labels to propose a new, paradigm-oriented taxonomy:
- Perception-Guided Heuristics: Mapping visual keypoints (like corners) to hand-crafted routines.
- Goal-Conditioned Policies: Training models to reach a specific target state defined by an image or coordinate.
- Predictive and Model-Based Methods: Learning explicit cloth dynamics or latent state representations for planning.
- Reward-Driven Reinforcement Learning (RL): Optimizing policies through trial-and-error using task-specific rewards.
- Demonstration-Driven Skill Transfer: Imitating expert human or "oracle" supervisor data.
- LLM-Based Planning: Leveraging Large Language Models for high-level task reasoning and sub-goal generation.
The Manipulation Pipeline and Primitive Actions
The standard robotic cloth manipulation pipeline involves a closed-loop process of observation (RGB and Depth), decision-making via a deep learning model, and the execution of primitive actions. We are currently seeing a shift from slow "Pick and Place" (P&P) actions toward dynamic primitives, such as "Dynamic Flinging," which uses momentum to unfold cloth rapidly and efficiently.
Datasets and the Sim2Real Gap
Data collection remains a major bottleneck. While 65.9% of studies rely on simulation (using tools like SoftGym or MuJoCo), these environments often lack realistic textile-air interactions. Real-world data collection is hindered by labeling difficulties, leading researchers to use innovative solutions like Labels from UltraViolet (LUV)—transparent fluorescent paint visible only under UV light—to automate ground-truth collection. The "Sim2Real" gap—differences in stiffness, friction, and visual appearance—remains the primary hurdle for deploying laboratory-trained policies in the real world.
Key Results and Metrics
- Unfolding: Benchmarked primarily by cloth coverage (percentage of maximum area) and the number of action steps.
- Folding: Less standardized, often relying on Mean Intersection over Union (MIoU) or qualitative visual consistency.
- Hardware: Dual-arm systems (e.g., UR5 or Franka Panda) are becoming the standard for complex bimanual strategies.
Future Perspectives
To advance the field of Physical AI, future research must focus on:
- Task Integration: Moving toward unified end-to-end pipelines that handle cloth from a random crumpled state to a finished fold.
- Complex Garments: Expanding beyond towels to handle long sleeves, collars, and inside-out garments.
- Multimodal Sensing: Incorporating tactile and force feedback to detect layers and prevent grasp failures.
- Standardization: Establishing consensus metrics to allow for fair, quantitative comparisons across the community.
Summary updated