A Tutorial on Shape Servoing of Deformable Objects-Part II: Advanced Concepts

Ignacio Cuiral-Zueco, Alessio Caporali, Miguel Aranda, Gianluca Palli, Gonzalo López-Nicolás, Youcef MezouararXiv 2026 · arXiv

TL;DR

This paper provides a deep dive into the mechanics of Jacobian-based shape servoing for deformable objects. By applying Singular Value Decomposition (SVD) to the deformation Jacobian, the authors reveal how to diagnose control failures, choose optimal shape features to minimize object strain, and extend the framework to handle moving targets and secondary tasks in 3D space.

What Problem Does This Paper Address?

Manipulating deformable objects—like cables, fabrics, or tissues—is a major challenge in robotics because these objects have essentially infinite degrees of freedom. Traditional model-based control requires complex physical simulations that are often too slow or inaccurate for real-time use.

Data-driven "shape servoing" bypasses this by estimating a deformation Jacobian: a local linear model that relates robot gripper motion to changes in the object's observed shape. While Part I of this tutorial established the basic control loop, Part II addresses the "why" and "how" of advanced manipulation:

  • How do we know which shapes the robot can actually produce?
  • How does the choice of visual features (points vs. angles) change the way the object deforms?
  • Why does the controller sometimes fail (e.g., when a wire goes taut or buckles)?

Methods

The authors build upon the standard shape-servoing loop (shown in Figure 1) using several key analytical tools:

  • Singular Value Decomposition (SVD): They decompose the estimated Jacobian $\hat{\mathbf{J}}$ into singular triplets. This identifies "robot-motion directions" (coordinated movements of multiple grippers) and their corresponding "feature directions" (how the shape changes).
  • Feature Engineering: The paper compares three levels of shape representation:
  • Point Coordinates: Extrinsic features that track absolute positions.
  • Edge Vectors: Translation-invariant features.
  • Turning Angles: Intrinsic features invariant to both translation and rotation.
  • Null-Space Projection: For redundant systems (where the robot has more degrees of freedom than the shape task requires), the authors use the Jacobian's null space to perform secondary tasks—like moving a gripper to a specific location—without disturbing the object's primary shape.
  • Failure Mode Analysis: They use a chain-rule factorization of the Jacobian to distinguish between controllability failures (the robot can't move the object further) and observability failures (the sensors can't see the movement).
Fig. 1. Shape-servoing loop of Part I applied to the running example. Left: the controller sends the command uk of (2) to the robots, perception extracts the features sk from the camera measurement yk, and the Jacobian update corrects the estimate with (3) from the latest action–response pair, starting from the offline estimate ˆJ0. Right: two robot arms deform an elastic rod x (blue), observed by a stationary camera, toward a target shape x∗(dashed). The arrows at the grippers indicate the six in-plane actuation DoFs q = [xL, yL, θL, xR, yR, θR]⊤.

Datasets

The paper utilizes simulated environments to provide reproducible "running examples":

  • 2D Elastic Rod: A Kirchhoff rod model discretized into 25 nodes, manipulated by two 3-DOF grippers.
  • 3D Objects: Simulations of a 3D rod, a triangular surface mesh (sheet), and a volumetric lattice (solid) using "As-Rigid-As-Possible" (ARAP) deformation models.
  • Interactive Demos: The authors provide public code and web-based interactive demos to reproduce the paper's numerical results.

Key Insights and Ideas

  • The "Cost" of Feature Choice: The path an object takes to a target shape depends heavily on the features used. Controlling via point coordinates can lead to high internal stress (e.g., stretching a rod), whereas curvature-based features allow the object to "relax" into the target shape more naturally.
  • SVD as a Diagnostic: The singular values $\sigma_l$ quantify the "gain" of the system. A very small $\sigma_{min}$ indicates a "weak" direction where the robot must move a great distance to achieve a tiny shape change, often leading to unstable, jerky commands.
  • Mechanical Instability as a Singularity: The paper highlights that physical events like buckling (shown in Figure 5) represent a loss of regularity in the equilibrium mapping, which manifests as a failure in the local Jacobian model.
  • Feedforward for Fluidity: By adding a feedforward term that accounts for target velocity, the controller can track moving shapes with near-zero lag, whereas pure feedback control always suffers from a "tracking lag."

Key Results

  • Strain Reduction: In a "looped rod" experiment, using curvature-based features kept the maximum axial strain at 2.1%, while point-coordinate features forced the rod through a high-stress path peaking at 27% strain.
  • Tracking Accuracy: For a time-varying target, adding a feedforward term reduced the peak shape error from 0.46 to less than $1.7 \times 10^{-3}$ (Figure 6).
  • Robustness to Disturbance: The authors demonstrate that intrinsic features (turning angles) allow the robot to ignore rigid-body disturbances (like someone bumping the table) that would otherwise cause a point-based controller to fight to return to an absolute coordinate.
Fig. 2. Offline Jacobian estimate of the running example and its SVD. Left: signed entries of ˆJ0; columns correspond to the coordinates of the left (violet) and right (orange) grippers, rows to the image coordinates of the tracked points. Right: one row per singular triplet, from the largest to the smallest singular value. Each row shows the singular value σl (the directional input–output gain) on a logarithmic axis, the robot direction ψl over the six actuation directions, and the feature direction ϕl, drawn on the rod as one arrow per tracked point (yellow). Their relationship is ˆJ0ψl = σlϕl, where the components of ψl weight the columns of the Jacobian estimate, and ϕl gives the resulting feature direction with magnitude scaled by σl. All rows use the same arrow scale.

Limitations and Future Work

  • Material History: The current Jacobian model assumes the object is quasistatic and elastic. It does not account for plasticity (permanent deformation) or history-dependent materials, which would require more sophisticated adaptive models.
  • High-Speed Dynamics: The tutorial focuses on slow, controlled movements. "Dynamic shape servoing" that accounts for inertia and damping remains a frontier.
  • Complex Topology: The methods assume a consistent "domain map" (correspondence between points). Handling objects that break, join, or undergo extreme occlusions remains a significant perception challenge.

Summary updated

Abstract

In robotics, shape control, or shape servoing, refers to the automatic manipulation of a deformable object toward a desired shape. This article is the second of a two-part tutorial on data-driven, Jacobian-based shape control, in which the deformation Jacobian, a local model of how robot motion changes the measured shape, is estimated from measurements. Part I built the shape-servoing loop and derived a local stability condition. This paper starts by using the singular value decomposition of the deformation Jacobian to identify shape changes that can be produced by small robot motions, those that require large robot motions, and those that are locally unavailable. It then analyses how the choice of shape features determines how the controller regulates the object's shape and, therefore, how much the object deforms on its way to the target. It then describes three ways in which the shape control loop can fail: loss of controllability, loss of observability, and occurrence of mechanical instabilities. It closes with shape-control extensions to time-varying targets, secondary tasks, and three-dimensional objects.
BibTeX
@misc{cuiralzueco2026tutorial,
  author = {Ignacio Cuiral-Zueco and Alessio Caporali and Miguel Aranda and Gianluca Palli and Gonzalo López-Nicolás and Youcef Mezouar},
  title = {{A Tutorial on Shape Servoing of Deformable Objects-Part II: Advanced Concepts}},
  year = {2026},
  eprint = {2611.03825},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.03825},
}
Figure