In classical robotics, we often rely on a "rigid body assumption." We assume that once a gripper closes on a tool, the tool becomes an extension of the robot’s arm—a fixed geometric entity. We also assume that when a robot probes an object to sense its shape, that object remains rooted to the spot.
However, as Drew Bagnell and colleagues noted in Efficient Touch Based Localization through Submodularity, this is rarely true in the real world. Physical interaction is inherently disruptive. The very act of "feeling" an object often pushes it away, and the high forces required for assembly often cause a grasped part to pivot or slide within the fingers. This research direction focuses on breaking the rigid-body myth, developing perception and control systems that can "expect" and compensate for unintended object movement.
The Fragility of the Rigid Grasp
The most common failure point in precision manipulation is in-hand slippage. When a robot performs a contact-rich task—like inserting a peg or threading a nut—the resistance from the environment often exceeds the friction of the grip.
As Byron Boots and Mustafa Mukadam explore in Neural Contact Fields and ViTaSCOPE, if an object shifts even a few millimeters inside the gripper, the robot’s internal "map" of the tool is instantly invalidated. The robot thinks the tip of the tool is at coordinate $A$, but it has slipped to coordinate $B$. Without tactile feedback to track this internal state transition, the robot continues to apply force based on a ghost pose, leading to what Bingjie Tang and Dieter Fox describe in IndustReal as a total loss of pose stability.
This problem isn't just about linear sliding. Dimitrios Dimou points out in Force Feedback Control For Dexterous Robotic Hands that rotational slips—where an object pivots between the fingertips—are particularly difficult to detect and stabilize, yet they are ubiquitous when handling heavy or unbalanced objects.
The Observer's Paradox in Tactile Sensing
A similar problem occurs during exploration. To understand the shape of an unknown object, a robot must touch it. But as Guillaume Duret highlights in Tactile MNIST, the force of a tactile sensor often causes lightweight or unanchored objects to shift, rotate, or tumble.
This creates a "spatial misalignment" problem. In ShapeMap 3-D, Sudharshan Suresh and Zilin Si explain that if an object moves between two successive touches, the robot cannot easily fuse those two "imprints" into a single 3D model. The second touch is recorded in a coordinate frame that no longer matches the first. This violation of the static-world assumption is a primary reason why tactile reconstruction often lags behind visual reconstruction in accuracy.
Why the Math Breaks
When objects move unexpectedly, the underlying mathematical frameworks used for control and estimation begin to fail in predictable ways:
- Kinematic Corruption: In Online Estimation of Articulated Objects, Adrian Röfer notes that if a gripper slips while trying to open a door, the system interprets that slip as part of the door's articulation. The resulting kinematic model becomes "noisy" because it cannot distinguish between the object's intended motion and the robot's failed grip.
- Gradient Inaccuracy: Abdeslam Boularias shows in Identifying Mechanical Models of Unknown Objects that physics-based identification relies on a constant contact point. If the manipulator slides along the surface, the analytical gradients used to update the model point in the wrong direction, leading to incorrect estimates of mass and friction.
- Correspondence Loss: Dimitrios Tzionas and Juergen Gall demonstrate in 3D Object Reconstruction that slippage invalidates the spatial correspondences needed to infer motion from a manipulator's trajectory.
Emerging Solutions: From Sensing to Mechanical Intelligence
To solve these issues, researchers are moving toward "closed-loop" tactile perception. Instead of assuming a grasp is stable, systems like The Feeling of Success by Roberto Calandra use continuous GelSight feedback to predict and detect slips before they lead to catastrophic failure.
Others are looking at "mechanical intelligence." Eric C. Joyce argues in Is Image-based Object Pose Estimation Ready to Support Grasping? that we shouldn't just rely on better algorithms. Underactuated, compliant hands can physically "absorb" the errors caused by slippage and displacement, establishing stable contacts even when the robot's internal pose estimate is slightly off.
Ultimately, the goal is to move from a "static" view of the world to a "dynamic" one, where the robot treats every interaction as a potential source of movement, constantly re-aligning its internal models with the shifting reality of the objects it touches.