MorphTwin: Enriching Deformable Material Responses from Videos for Morphable and Interactive Physical Twins

Haoqin Hong, Ding Fan, Fubin Dou, Tianyu Huang, Yuzhuo Chen, Zhemeng Zhang, Hanxiao Jiang, Congcong Zhu, Gerhard Petrus Hancke, Jingrun Chen, Rynson Wh LauarXiv 2026 · arXiv

TL;DR

MorphTwin is a novel framework that creates physically accurate, interactive digital twins of deformable objects directly from sparse video. By introducing "response carriers" into traditional mass-spring models, it decouples volume and shape responses, allowing for precise material identification and interpretable editing while maintaining the real-time simulation speeds required for VR and robotics.

What Problem Does This Paper Address?

Creating "physical twins"—digital replicas that not only look like real-world objects but also react realistically to new forces—is a holy grail for spatial computing and embodied AI. While methods like Gaussian Splatting have mastered visual reconstruction, capturing the underlying physics remains difficult.

The authors identify a fundamental bottleneck in current real-time physics models: structural coupling. Most efficient simulations rely on 3D central-force spring networks. However, these networks are mathematically restricted; they cannot independently control how an object changes volume (bulk modulus) versus how it changes shape (shear modulus). This leads to a fixed Poisson’s ratio (typically $\nu = 0.25$), meaning a digital sponge might stretch correctly but fail to "thin out" realistically. Consequently, these models often fail to generalize when subjected to forces different from those seen in the training video.

Methods

MorphTwin enriches the expressive capacity of mass-spring systems without sacrificing their efficiency. Its core innovations include:

  1. Relaxable Response Carriers: Instead of a simple grid of springs, MorphTwin augments the object's graph with internal auxiliary nodes. These "carriers" can move relative to the surrounding nodes, providing the "non-affine" freedom needed to model complex deformations. Specifically, it uses separate carrier structures for volumetric (bulk) and deviatoric (shear) responses.
  2. Energy-Based Calibration: Spring parameters are notoriously dependent on the specific geometry and connectivity of a mesh. MorphTwin uses an offline calibration step to map high-level material properties (like Young’s modulus and Poisson’s ratio) to specific spring stiffness updates for a given graph. This allows for "material morphing"—making a reconstructed hat stiffer or a book softer—without refitting the entire model.
  3. Two-Stage Fitting: The pipeline first fits the physical parameters (stiffness, friction, etc.) to the observed motion in the video. Once the physics are locked, it optimizes a Gaussian Splatting-based appearance model on top of the physical particles.
The MorphTwin pipeline: from sparse video to a calibrated, interactive physical twin.

Datasets

The authors evaluated MorphTwin across several benchmarks:

  • PhysTwin Benchmark: 20 scenes used in prior work for deformable object reconstruction.
  • Spring-Gaus: 14 synthetic scenes with known ground-truth material properties to validate material identification accuracy.
  • New Egocentric Benchmark: The authors introduced a new dataset of 15 sequences featuring temporally aligned egocentric stereo (from a Meta Quest 3) and external RGB-D observations of real-world interactions with objects of varying stiffness.

Key Insights and Ideas

  • Breaking the Cauchy Restriction: Standard spring networks are bound by the "Cauchy restriction," which couples bulk and shear moduli. By allowing internal relaxation via auxiliary nodes, MorphTwin effectively breaks this mathematical limit, as illustrated in Figure 3.
  • Material Reuse is a Mapping Problem: You cannot simply copy a "stiffness" value from one digital object to another if their underlying meshes are different. MorphTwin’s "shared response space" acts as a universal translator, allowing a user to define a material change once and apply it accurately to any reconstructed asset.
  • Explicit is Better for Interaction: While complex methods like the Material Point Method (MPM) offer high fidelity, they are often too slow for real-time VR. MorphTwin proves that you can get "MPM-like" material expressiveness while staying within the fast, explicit integration framework of mass-spring systems.
How response carriers decouple volume and shear modeling to improve material identification.

Key Results

MorphTwin significantly outperforms existing baselines in both physical accuracy and visual fidelity:

  • Material Accuracy: On synthetic tests, it reduced the mean absolute error of Poisson’s ratio by 75.8% and the mean relative error of Young’s modulus by 22.1% compared to PhysTwin.
  • Predictive Power: In real-world egocentric sequences, MorphTwin achieved up to 14.4% lower future tracking error, meaning it better predicts how an object will move in frames it hasn't seen yet.
  • Interpretable Editing: The system allows for intuitive material edits. For example, a user can take a reconstructed neck pillow and make it 10x stiffer or take a rigid book and make it soft enough to bend, all while preserving the original visual appearance.
Qualitative results showing material editing (stiffening/softening) on real-world reconstructed assets.

Limitations and Future Work

While MorphTwin is a major step forward, the authors note a few areas for growth:

  • Isotropy: The current formulation focuses on isotropic materials (those that behave the same in all directions). Future versions could extend this to anisotropic materials like wood or muscle tissue.
  • Fixed Density: The model currently assumes a fixed density for objects. Jointly identifying mass and stiffness from visual data remains a challenging "ill-posed" problem.
  • Speed of Creation: The reconstruction process still involves an optimization loop. The authors suggest that future work could integrate feed-forward neural networks to "guess" initial physical parameters, further streamlining the creation of physical twins.

Summary updated

Abstract

Reconstructing interactive physical twins from video requires accurately capturing how objects deform while maintaining efficient simulation. We observe that reproducing motion in a video does not guarantee correct responses to new forces when the physical representation restricts how an object can deform. In spring-based twins, coupling between volume and shape responses creates such a limitation. We present MorphTwin, which enriches local deformation modes to capture and independently control these responses while retaining standard spring forces and explicit simulation. To support interpretable material editing and transfer, we use offline calibration to map shared relative response changes to stiffness updates on each reconstructed graph. Under identical inverse settings on synthetic scenes, MorphTwin reduces the mean absolute error of Poisson's ratio by 75.8% and the mean relative error of Young's modulus by 22.1% compared to PhysTwin. We introduce an egocentric stereo and RGB-D benchmark covering interactions with objects of varying stiffness. Real-world experiments demonstrate improved reconstruction accuracy and up to 14.4% lower future tracking error than PhysTwin at interactive speeds.
BibTeX
@misc{hong2026morphtwin,
  author = {Haoqin Hong and Ding Fan and Fubin Dou and Tianyu Huang and Yuzhuo Chen and Zhemeng Zhang and Hanxiao Jiang and Congcong Zhu and Gerhard Petrus Hancke and Jingrun Chen and Rynson WH Lau},
  title = {{MorphTwin: Enriching Deformable Material Responses from Videos for Morphable and Interactive Physical Twins}},
  year = {2026},
  eprint = {2611.25963},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.25963},
}
Figure