Phy-Tac: Toward Human-Like Grasping via Physics-Conditioned Tactile Goals

Shipeng Lyu, Lijie Sheng, Fangyuan Wang, Wenyao Zhang, Weiwei Lin, Zhenzhong Jia, David Navarro-Alarcon, Guodong GuoarXiv 2025 · DOI · arXiv

Abstract

Traditional robotic grasping often relies on excessive force to ensure stability, leading to potential damage to fragile objects and energy inefficiency. This paper introduces Phy-Tac, a framework that moves robotic manipulation away from reactive "over-squeezing" toward human-like, minimal-force stability. By utilizing a Physics-conditioned Latent Diffusion Model (Phy-LDM), the system proactively "hallucinates" optimal tactile imprints for specific objects and regulates gripping force via a latent-space LQR controller. Experimental results demonstrate a 30–40% reduction in tactile prediction error and significantly improved force efficiency across diverse object categories.

Introduction

Most robots today employ a "better safe than sorry" approach to grasping, applying excessive force to prevent slippage. While effective for rigid industrial parts, this strategy fails when handling fragile or compliant objects like fruit and bread. Humans, by contrast, use an anticipatory process: we estimate required force based on appearance and refine it using tactile feedback to reach a "just-enough" level of grip. Phy-Tac addresses the challenge of unifying pose planning, tactile sensing, and force control into a single, proactive human-inspired loop.

Comparison of human and human-inspired robotic grasping strategies across three stages: pose planning (S1), force prediction (S2), and control (S3).

Methodology

The Phy-Tac framework consists of three main stages designed to achieve Force-Optimal Stable Grasping (FOSG).

1. Contact-Optimized Pose Planning

Rather than selecting grasps based solely on model confidence, Phy-Tac filters candidates using a physics-inspired geometric consistency metric. It prioritizes surface regions with high roughness for friction, consistent normals for alignment, and uniform curvature to avoid stress concentrations.

2. Physics-conditioned Latent Diffusion Model (Phy-LDM)

The core innovation is the Phy-LDM, which generates a high-fidelity "target" tactile image based on the object's mass, texture, and contact patch geometry. This image represents the ideal sensor state for a stable grasp, allowing the robot to set a tactile goal before the grasp is completed.

3. Latent-Space LQR Servo

To reach the predicted goal, the robot uses a Linear Quadratic Regulator (LQR) operating within the compressed latent space of a Variational Autoencoder (VAE). The controller minimizes the distance between current tactile sensations and the predicted goal, stopping exactly when the minimal necessary force is reached.

The technical architecture of the Phy-Tac framework, detailing the SC Dataset components and the three-stage pipeline for force-optimal stable grasping.

Datasets

The authors introduced the SC Dataset to provide the physical metadata necessary for force reasoning. It includes 34 diverse objects (tools, fruits, vegetables) with ~8.6K paired entries. Features include 3D meshes, object mass, material labels, and synchronized RGB tactile images paired with gripper commands.

Key Results and Insights

  • Superior Prediction: Phy-LDM reduced tactile image prediction errors (MAE and RMSE) by 30–40% compared to standard Latent Diffusion Models and outperformed GAN-based methods.
  • Force Efficiency: Phy-Tac achieved a higher FOSG rate than baselines, identifying minimal stability thresholds and using significantly less force than fixed-force strategies.
  • Geometry Dominance: Ablation studies revealed that contact geometry is more critical for accurate tactile prediction than mass or material properties.

Conclusion and Future Work

Phy-Tac demonstrates that diffusion models are highly effective at modeling complex tactile deformations for proactive force control. While the system generalizes well to rigid objects, handling highly deformable items remains a challenge due to changing geometry during contact. Future research will focus on dynamic updates to physics priors to accommodate real-time object deformation.

Summary updated

Abstract

Humans naturally grasp objects with minimal level required force for stability, whereas robots often rely on rigid, over-squeezing control. To narrow this gap, we propose a human-inspired physics-conditioned tactile method (Phy-Tac) for force-optimal stable grasping (FOSG) that unifies pose selection, tactile prediction, and force regulation. A physics-based pose selector first identifies feasible contact regions with optimal force distribution based on surface geometry. Then, a physics-conditioned latent diffusion model (Phy-LDM) predicts the tactile imprint under FOSG target. Last, a latent-space LQR controller drives the gripper toward this tactile imprint with minimal actuation, preventing unnecessary compression. Trained on a physics-conditioned tactile dataset covering diverse objects and contact conditions, the proposed Phy-LDM achieves superior tactile prediction accuracy, while the Phy-Tac outperforms fixed-force and GraspNet-based baselines in grasp stability and force efficiency. Experiments on classical robotic platforms demonstrate force-efficient and adaptive manipulation that bridges the gap between robotic and human grasping.
BibTeX
@misc{lyu2025phy,
  author = {Shipeng Lyu and Lijie Sheng and Fangyuan Wang and Wenyao Zhang and Weiwei Lin and Zhenzhong Jia and David Navarro-Alarcon and Guodong Guo},
  title = {{Phy-Tac: Toward Human-Like Grasping via Physics-Conditioned Tactile Goals}},
  year = {2025},
  eprint = {2511.01520},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2511.01520},
}
Figure