Deep learning-based robotic cloth manipulation applications: systematic review, challenges and opportunities for physical AI

Ningquan Gu, Mitsuhiro Hayashibe, Kyo Kutsuzawa, Hui YuFrontiers in Robotics and AI 2026 · DOI

Abstract

Cloth unfolding and folding are fundamental tasks in autonomous robotic cloth manipulation, representing a significant frontier in Physical AI. Driven by recent advances in deep learning, this field has expanded rapidly. This systematic review identifies and summarizes progress in deep learning-based cloth manipulation, following PRISMA guidelines to analyze 41 relevant papers published between 2019 and 2024. While current methods show impressive performance, challenges remain regarding irregular cloth sizes, diverse initial states, and the "Sim2Real" gap. This paper proposes a new taxonomy of six learning and control paradigms and discusses the future of multi-modal sensing and standardized metrics in the field.

Introduction

Cloth manipulation is often considered a "grand challenge" for robotics. Unlike rigid objects, cloth possesses an infinite-dimensional configuration space, complex non-linear dynamics, and a tendency for constant self-occlusion. While deep learning (DL) has accelerated progress, the research landscape has remained fragmented. This review provides a structured analysis of task contents, datasets, and primitive actions to offer a roadmap for future research in Physical AI.

Methodology

To ensure a rigorous and reproducible analysis, the authors followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. The selection process involved filtering 655 records from major databases (IEEE Xplore, Scopus, Web of Science, and ACM Digital Library) down to 41 high-quality papers based on strict inclusion criteria, such as peer-review status and a focus on deep learning-based unfolding or folding.

Flowchart of the literature selection process following PRISMA guidelines.

The Six Paradigms of Robotic Cloth Manipulation

The review moves beyond simple "supervised vs. reinforcement learning" labels to propose a new, paradigm-oriented taxonomy:

  1. Perception-Guided Heuristics: Mapping visual keypoints (like corners) to hand-crafted routines.
  2. Goal-Conditioned Policies: Training models to reach a specific target state defined by an image or coordinate.
  3. Predictive and Model-Based Methods: Learning explicit cloth dynamics or latent state representations for planning.
  4. Reward-Driven Reinforcement Learning (RL): Optimizing policies through trial-and-error using task-specific rewards.
  5. Demonstration-Driven Skill Transfer: Imitating expert human or "oracle" supervisor data.
  6. LLM-Based Planning: Leveraging Large Language Models for high-level task reasoning and sub-goal generation.

The Manipulation Pipeline and Primitive Actions

The standard robotic cloth manipulation pipeline involves a closed-loop process of observation (RGB and Depth), decision-making via a deep learning model, and the execution of primitive actions. We are currently seeing a shift from slow "Pick and Place" (P&P) actions toward dynamic primitives, such as "Dynamic Flinging," which uses momentum to unfold cloth rapidly and efficiently.

Overview of the robotic cloth manipulation pipeline: from observation to performance metrics.

Datasets and the Sim2Real Gap

Data collection remains a major bottleneck. While 65.9% of studies rely on simulation (using tools like SoftGym or MuJoCo), these environments often lack realistic textile-air interactions. Real-world data collection is hindered by labeling difficulties, leading researchers to use innovative solutions like Labels from UltraViolet (LUV)—transparent fluorescent paint visible only under UV light—to automate ground-truth collection. The "Sim2Real" gap—differences in stiffness, friction, and visual appearance—remains the primary hurdle for deploying laboratory-trained policies in the real world.

Key Results and Metrics

  • Unfolding: Benchmarked primarily by cloth coverage (percentage of maximum area) and the number of action steps.
  • Folding: Less standardized, often relying on Mean Intersection over Union (MIoU) or qualitative visual consistency.
  • Hardware: Dual-arm systems (e.g., UR5 or Franka Panda) are becoming the standard for complex bimanual strategies.

Future Perspectives

To advance the field of Physical AI, future research must focus on:

  • Task Integration: Moving toward unified end-to-end pipelines that handle cloth from a random crumpled state to a finished fold.
  • Complex Garments: Expanding beyond towels to handle long sleeves, collars, and inside-out garments.
  • Multimodal Sensing: Incorporating tactile and force feedback to detect layers and prevent grasp failures.
  • Standardization: Establishing consensus metrics to allow for fair, quantitative comparisons across the community.

Summary updated

Abstract

Cloth unfolding and folding are fundamental tasks in autonomous robotic cloth manipulation as Physical AI. Driven by recent advances in deep learning, this area has developed rapidly in recent years. This review aims to systematically identify and summarize current progress in deep learning-based cloth unfolding and folding. Following the Systematic Reviews and Meta-Analyses (PRISMA) guidelines, 41 relevant papers from 2019 to 2024 were selected for analysis. We examines various factors influencing cloth manipulation and find that, while current methods show impressive performance, several challenges remain unaddressed. These challenges include irregular cloth sizes and diverse initial garment states. Concerning datasets, there is a need for improved real-world data collection systems and more realistic cloth simulators, and the Sim2Real gap must be carefully considered. Additionally, the review highlights the importance of incorporating multi-modal sensors into current platforms and the emergence of novel primitive actions that enhance performance. The need for more consistent comparison metrics is emphasized, and strategies for addressing failure modes are discussed to further advance the field. From an algorithmic perspective, we reorganize existing learning methods into six learning and control paradigms: perception-guided heuristics, goal-conditioned manipulation policies, predictive and model-based state representation methods, reward-driven reinforcement learning over primitive actions, demonstration-driven skill transfer methods, and emerging large language model-based planning methods. We discuss how each paradigm contributes to unfolding and folding, their respective strengths and limitations, and the open problems that arise. Finally, we summarize the remaining challenges and provide future perspectives for physical AI.
BibTeX
@inproceedings{gu2026deep,
  author = {Ningquan Gu and Mitsuhiro Hayashibe and Kyo Kutsuzawa and Hui Yu},
  title = {{Deep learning-based robotic cloth manipulation applications: systematic review, challenges and opportunities for physical AI}},
  booktitle = {Frontiers in Robotics and AI},
  year = {2026},
  doi = {10.3389/frobt.2026.1752914},
}
Figure