Robotic Manipulation of Deformable Objects

72 concepts71 papers1 proposal

Traditional robotics excels at manipulating rigid objects because their state can be perfectly captured by just six degrees of freedom: position and orientation along the X, Y, and Z axes. If a robotic gripper holds a steel wrench, every atom of that wrench moves predictably in lockstep with the robot's arm.

However, this rigid-body assumption completely breaks down when dealing with flexible materials like ropes, fabrics, plasticine, or food. These items possess effectively infinite degrees of freedom and exhibit highly non-linear, under-actuated physical behaviors. Pushing a rope does not move it forward; it crumples it. Tossing a semi-fluid ingredient in a wok alters its shape entirely. Because these objects do not maintain a fixed geometry, traditional analytical control models become computationally intractable or wildly inaccurate.

To bridge this gap, modern research focuses on three core pillars: learning low-dimensional topological representations for perception, leveraging differentiable and hybrid simulations to predict material behavior, and utilizing dynamic motion primitives to bypass complex geometric modeling.

Representing the Unrepresentable: Perception and State Estimation

How do you track an object that changes shape with every touch? A primary hurdle in computer vision for soft bodies is distinguishing functional features from transient deformations. For instance, when a robot interacts with cloth, Brian Okorn et al. highlight that standard vision algorithms struggle to separate critical grasping features (like edges and corners) from random wrinkles and folds.

To make perception tractable, researchers avoid dense, full-mesh tracking in favor of topological or keypoint representations. When dealing with Deformable Linear Objects (DLOs) like wiring harnesses or medical catheters, Bare Luka Zagar et al. demonstrate that models can successfully generalize across entirely different domains (such as automotive wiring versus surgical blood vessels) by focusing on shared topological features rather than exact geometric configurations.

[Dense 3D Mesh] ----(Too many degrees of freedom)----> Intractable Control
[Topological Keypoints] --(Edges, Corners, Knots)----> Tractable Control

Visual data alone, however, is frequently compromised by severe self-occlusion; a crumpled plastic bag or a folded garment hides its own structure. To overcome this visual limitation, Youngsun Wi et al. combine partial visual tracking with wrist-mounted force sensors. By learning a latent representation of visuo-tactile dynamics, their system can infer the complete 3D geometry of an object even when it is heavily occluded by the robot's own gripper.

Predicting Material Behavior: Differentiable Physics and Sim-to-Real

Simulating soft-body dynamics requires immense computational resources. As noted by Russell Gayle, simulating behaviors like bending and twisting while maintaining volume constraints traditionally creates a massive bottleneck in motion planning. Furthermore, standard physics engines cannot efficiently compute the analytical gradients needed for rapid reinforcement learning when materials undergo plastic (permanent) or elastic deformation.

To accelerate learning, frameworks like PlasticineLab, introduced by Zhiao Huang et al., utilize fully differentiable physics engines. By providing direct gradient information, these simulators allow robots to learn complex shaping tasks—such as molding clay into target geometries—in a fraction of the time required by traditional model-free reinforcement learning.

Yet, even the best simulations suffer from the "sim-to-real gap" because real-world friction, material fatigue, and micro-interactions are incredibly difficult to formalize. To address this, researchers deploy hybrid offline-online architectures. For example, Changhao Wang et al. use a Graph Neural Network (GNN) trained in simulation to capture coarse, global cable dynamics offline, while running a lightweight linear residual model online to correct for physical discrepancies in real time.

Similarly, Shixin Zhang et al. integrate vision-based tactile sensors into both the simulated and physical environments. By aligning the simulated tactile feedback with real-world sensor readings, the authors achieve highly reliable sim-to-real transfer for high-precision tasks like shaping plasticine into cylinders and spheres.

Bypassing Complexity via Dynamic Actions and Primitives

Because predicting the exact continuous state of a flexible object during slow, quasi-static movements is so difficult, an elegant alternative has emerged: dynamic manipulation. Instead of fighting the object's flexibility, robots can exploit fluid-structure interactions and momentum to force the material into predictable states.

A clear example of this intuition is found in ShakingBot, developed by Ningquan Gu et al.. Trying to open a crumpled plastic bag by carefully separating its edges with a gripper is an open perception nightmare. The authors bypass this entirely by using high-speed shaking actions and airflow, using the bag's own aerodynamic drag to separate the layers and expose the opening.

Quasi-static approach: Track every wrinkle -> Attempt precise pinch -> High failure rate
Dynamic approach: High-speed shaking -> Aerodynamic drag forces opening -> High success rate

For high-speed cable manipulation, Kuniyuki Takahashi combines a stochastic forward model with black-box optimization to teach a robot to whip a cable into a target shape. The rapid, ballistic motion makes the cable's trajectory highly dependent on momentum, allowing the system to clear local minima that typically trap slower, gradient-based control strategies.

When tasks demand geometric precision rather than raw speed—such as knot tying or mechanical assembly—researchers abstract the continuous action space into discrete, high-level primitives. Mengyuan Yan et al. decompose intricate knotting tasks into "Topological Motion Primitives," combining abstract knot theory with reinforcement learning to plan paths that human operators can easily demonstrate. For industrial settings, Shiyu Jin et al. utilize Mathematical Programming with Complementarity Constraints (MPCC) to automatically discover the optimal trajectory for assembling flexible belt drives, seamlessly handling the sudden transition when a belt shifts from slack to taut.

Coordinated and Multi-Agent Control

Finally, because deformable objects can bend around obstacles or slip out of single points of contact, many tasks inherently require coordinated multi-arm manipulation.

In the domain of robotic cooking, Junjia Liu et al. manage the unpredictable deformation of semi-fluid food objects by decoupling a bimanual system into a "leader" arm (controlling the wok) and a "follower" arm (controlling the spatula). Their movements are synchronized via a specialized Structured-Transformer that models the non-prehensile tossing and rolling of the food.

Taking coordination a step further, Shuaixing Chen demonstrates that passive flexible materials can be treated as active components within a multi-agent framework. In a robot-assisted patient transfer scenario, the system uses multi-agent model predictive control to autonomously coordinate the rigging and tying of deformable sling straps to moving hooks, proving that understanding flexible material dynamics is essential for safe, physical human-robot interaction.

Key Foundational Work

Quasi-static manipulation of a planar elastic rod using multiple robotic grippers (2014) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_deformable_object_manipulation_deformabl_8290eef1,
  title = {deformable object manipulation (deformable object manipulation challenges)},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/deformable-object-manipulation-deformable-object-manipulation-challenges/rdr_8290eef143b1496f83a7e07594aff933},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 1, frozen text):

@misc{lacuna_deformable_object_manipulation_deformabl_8290eef1_v1,
  title = {deformable object manipulation (deformable object manipulation challenges)},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/deformable-object-manipulation-deformable-object-manipulation-challenges/rdr_8290eef143b1496f83a7e07594aff933/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake