Traditional robotics excels at moving rigid objects—engine blocks, circuit boards, or shipping crates—where the object’s state can be described by a simple six-degree-of-freedom (6-DOF) pose. However, as noted by Heni Ben Amor and Jan Peters in their foundational editorial, the transition to "human-level" manipulation requires robots to handle deformable objects like cloth, rope, and soft tissues.
The core challenge is that these materials possess effectively infinite degrees of freedom. A t-shirt doesn't just have a position; it has a near-infinite variety of wrinkles, folds, and self-occlusions. This makes the state space too vast for traditional geometric modeling and the dynamics too non-linear for standard control.
The Perception Gap: Beyond Rigid Poses
In rigid-body robotics, if you see three corners of a box, you know where the fourth is. With deformable objects, this logic collapses. Fabric and cables are prone to self-occlusion, where the object hides its own configuration from the sensor. Li Sun argues that overcoming this requires moving away from task-specific "hacks" toward hierarchical vision architectures that can perceive 3D structure even in cluttered states.
To track these objects, researchers have moved toward distributional and graph-based representations:
- Correspondence Matching: Aditya Ganapathi and Priya Sundaresan developed Multi-Modal Gaussian Shape Descriptors (MMGSD) to identify symmetric points (like the four identical corners of a towel) while providing a measure of uncertainty when parts are hidden.
- Topological Features: For 1D objects like wires or blood vessels, the PointWire and PointVessel datasets demonstrate that robots can learn shared topological features that generalize across vastly different domains, such as automotive wiring and medical surgery.
Learning Dynamics Without a Manual
Because the physics of a "floppy" object is so complex, writing down an analytical model is often impossible. Instead, the field has shifted toward learned dynamics.
Jia Pan and Zhe Hu demonstrated that robots can learn the relationship between their movements and an object's shape changes in real-time using Fast Online Gaussian Process Regression. This allows a robot to adapt to different materials—like switching from a heavy towel to a light silk—without a pre-defined model.
For more complex interactions, researchers use particle-based models. Xingyu Lin introduced Visible Connectivity Dynamics (VCD), which treats cloth as a graph of connected particles. By learning how these particles interact and reasoning about occluded connections, the robot can predict how pulling one corner will affect the rest of the fabric. Similarly, PlasticineLab by Zhiao Huang and Hao Su provides a differentiable physics environment where robots can use gradient-based optimization to solve manipulation tasks much faster than traditional trial-and-error reinforcement learning.
Bridging Simulation and Reality
Training a robot to fold clothes in the real world is slow and risks damaging hardware. The solution is often Sim-to-Real transfer. Jan Matas showed that by using heavy domain randomization—varying the colors, textures, and physics parameters in simulation—robots can learn policies that work immediately on real fabric.
However, simulation isn't always perfect. Changhao Wang and colleagues addressed this with a hybrid offline-online framework for cable manipulation. They use a Graph Neural Network (GNN) trained in simulation for the "big picture" movements, combined with a real-time residual model to correct for small physical discrepancies in the real world.
From Smoothing to Standardization
The ultimate goal is moving from simple "pick and place" to long-horizon tasks like laundry folding. This requires standardization—the ability to take a crumpled, unstructured heap and bring it into a canonical state.
Changshi Zhou and Feng Luan developed APS-Net, a framework that switches between high-speed "fling" actions to flatten garments and precise maneuvers to align them. Once standardized, complex tasks become manageable. For example, UniFolding uses a unified neural network called UFONet to handle both unfolding and folding across various garment types, while Foldsformer uses space-time attention to learn the sequential "instructions" of a fold from a single human demonstration.
By combining high-fidelity simulation, graph-based perception, and adaptive learning, the field is moving toward robots that don't just move objects, but truly understand the fluid, unpredictable nature of the deformable world.