Tissue Skating: Contact-Anchored 4D Reconstruction of Grasped Tissue

Animators have a name for a familiar bug in generated human motion. When a character's foot should be planted on the ground but drifts across it, that is "foot skating." Generative motion models produce it because they learn joint positions without any notion that a loaded foot must stay put. We argue that surgical scene reconstruction has the same bug in a form nobody has named or measured.

When a robotic grasper holds and pulls tissue, the grasped tissue must move exactly with the jaws. Current 4D tissue reconstructions do not enforce this. They cut the tool out of the video, so the tissue under the jaws is simply never observed, and the jaws drift relative to the tissue they are supposed to be holding. We propose to carry the foot-skating fix into surgical scene reconstruction with one essential change. The ground is stationary, but tissue is not, so the constraint must be "zero velocity relative to the moving tool" rather than "zero velocity in the world." We would then measure tissue skating directly, as a new failure metric.

The concrete system is a dynamic Gaussian-splatting reconstruction, meaning a scene represented by many small colored 3D blobs whose positions change over time and that can be rendered at high frame rate. We add a contact term driven by the robot's own joint readings. While the jaws are closed on tissue, the tissue blobs in the grasp patch are pulled to follow the jaws' rigid motion. A closed-form elastic falloff then spreads that motion into the surrounding hidden tissue. The payoff is reconstruction that is correct exactly where it matters for autonomy, at the point where the robot applies force, which is also where today's methods are blind.

Core Claim

The two research directions solve mirror-image halves of one problem. Motion synthesis knows how to enforce contact but assumes a rigid, motionless ground. Combining Recurrent Neural Networks and Adversarial Training for Human Motion Synthesis and Control predicts a binary contact label per foot and penalizes foot velocity whenever the label is on. Physics-filtering pipelines such as LAGOON and CRISP go further and track the motion in a rigid-body simulator. CRISP explicitly restricts the world to rigid planar boxes.

Deformable tissue reconstruction has the opposite profile. EndoGaussian, Endo-4DGS, EndoWave and LGS model non-rigid tissue motion well and fast. All four learn that motion purely from photometric and depth losses on tool-masked frames, and none has any concept of contact.

Our claim is that a relative contact invariant removes a measurable error that photometric losses cannot see, because the relevant pixels are occluded. That invariant is driven by robot kinematics, which are free on any robotic platform, and uses continuous rather than binary contact. The contribution sits at the reconstruction level, not in a simulator. We do not run a full physics engine. We use the tool as a moving, known boundary condition on the deformation field.

For the grasp patch, the invariant is the following loss:

$$ \mathcal{L}_{\text{grasp}} = \sum_t \sum_i c_t\, w_i \,\big\| \mathbf{x}_i(t{+}1) - T_{t+1} T_t^{-1}\, \mathbf{x}_i(t) \big\|^2 $$

The symbols mean the following:

  • $\mathbf{x}_i(t)$ is the center of tissue Gaussian $i$ at frame $t$.
  • $T_t$ is the rigid pose of the gripper jaws at frame $t$, obtained from forward kinematics after hand-eye calibration.
  • $c_t \in [0,1]$ is a soft grasp state, estimated from jaw opening angle and the depth gap between jaw and tissue.
  • $w_i$ is a weight that decays with distance from the jaw surface.

When $c_t = 0$ the loss vanishes. This recovers ordinary photometric reconstruction, so the method cannot be worse when no contact occurs. A one-sided hinge on signed distance to the jaw geometry adds non-penetration, which is the analogue of the foot not sinking into the floor.

Research Program

We would build on a HexPlane-style 4D Gaussian model in the EndoGaussian family, so that every result is a controlled modification of a strong existing system. The tool is represented separately, following Instrument-Splatting. That method binds Gaussians to the instrument's CAD parts and drives them by joint angles, but it treats tissue as background. We reuse its tool model unchanged and add the coupling that it omits.

Grasp-state estimation. The soft contact signal $c_t$ is the analogue of foot-contact labels. On robotic data it comes from the jaw-angle reading plus proximity. We will check its reliability against manually annotated grasp intervals, because a wrong contact label is the main way this method can hurt.

Falloff weights. The weights $w_i$ and the propagation of jaw motion into masked-but-not-grasped tissue use a Kelvinlet prior. Kelvinlets are the closed-form solution for how an infinite linear elastic medium displaces under a point load. We take this method from Neural-Augmented Kelvinlet. That work showed Kelvinlets work best as a regularizer on a learned deformation model rather than as the model itself. We use them exactly that way: as a soft prior on occluded Gaussians, with the photometric loss free to override it wherever tissue is visible. The paper's quasi-static, point-load assumptions constrain us as well. We therefore treat Kelvinlets only as an initialization and a regularizer for the occluded region, never as ground truth.

Tissue skating metric. We define tissue skating as the mean tangential velocity of reconstructed tissue at the jaw contact surface relative to the jaws, in mm/s, over grasp intervals. This is the direct analogue of foot-skating velocity in motion generation. A correct reconstruction of a non-slipping grasp should score near zero.

Baselines

Naive reference. The do-nothing reference is EndoGaussian run as published, with tool masking, on the same sequences, the same frame budget, and its default training schedule. The file reports its PSNR on EndoNeRF as 37.849 in EndoGaussian. Our method must stay within noise of that value on unoccluded pixels while reducing tissue skating and hidden-region depth error. A large PSNR drop would mean the contact term is distorting visible tissue.

Unmasked variant. We also run EndoGaussian without the tool mask, to test whether joint photometric learning alone captures contact.

Strongest competitor. The strongest competitor on this exact question is Real-to-Sim Registration of Deformable Soft Tissue with Position-Based Dynamics. That method couples dVRK tool kinematics to a position-based-dynamics tissue mesh registered to stereo point clouds, and it already fills in hidden tissue behind graspers. It reports reduced tracking error, particularly in depth, on pork-steak trajectories ("lift, cube, butterfly, sine wave"). No reported millimeter result was found on Lacuna for this method, so we will re-run it on the identical pork-steak sequences with its published uniform stiffness. We will also give it a stiffness grid search equal to our own hyperparameter budget.

Beating it means two things on held-out reveal frames: lower depth error in the formerly occluded region, and lower tissue skating. Rendering quality must also stay comparable to the 4D Gaussian baselines, which the mesh-based method does not provide.

Ablations. These isolate the key ingredient:

  • World-frame clamp: a zero-velocity clamp in world coordinates, i.e. the literal foot-skating loss. This tests whether relative velocity is what matters.
  • Hard rigid attachment: grasped Gaussians rigidly attached with no Kelvinlet falloff.
  • Kelvinlet-only: occluded displacement given by Kelvinlets with no grasp term.
  • Binary contact: binary instead of soft $c_t$.

Why not just…?

Why not just use the position-based-dynamics real-to-sim method, which already couples tool and tissue? This is the real threat, and we treat it as the main baseline rather than arguing it away. Its known limits are a static initial mesh, uniform pre-set stiffness, and perception from raw point clouds rather than a rendered field. These suggest it will lag on appearance and on noisy specular frames, but that is not evidence it loses on geometry under the jaws. If it matches us on reveal-frame depth error and tissue skating, the honest conclusion is that the contribution reduces to adding its constraint to Gaussian splatting. We would then report it that way.

Why not just stop masking the tool and let the 4D Gaussians learn tool and tissue together? Instrument-Splatting reports that EndoGaussian-style deformation fields reach only about 16–17 dB PSNR on articulated tools, against 23.84 dB for its kinematic tool model. A smooth deformation field therefore blurs the tool and tissue together instead of separating them. Our unmasked-EndoGaussian baseline tests this directly on the tissue side by measuring skating and penetration.

Why not just rigidly glue the grasped Gaussians to the jaws? This may well suffice inside the jaw patch, and that is why it is an ablation. Rigid gluing says nothing about the occluded ring around the jaws, where tissue stretches rather than translates. It also fails at grasp onset and release, where a hard switch causes popping. If hard attachment equals the full method on reveal-frame error in the ring, we would drop the Kelvinlet prior and keep the simpler design.

Why not just apply the standard foot-contact zero-velocity loss? In the world frame, that loss freezes grasped tissue while the robot is lifting it, which is precisely wrong during traction. The world-clamp ablation should show large errors on lift trajectories. Tissue-skating values that look acceptable would expose the clamp only through high reveal-frame depth error, which is why we report both measures.

Proposed Experiments and Decisive Checks

Primary data. The primary data are the dVRK chicken-skin and pork-steak sequences from SuPer and the follow-up position-based-dynamics work, because they include tool kinematics. Hidden-region ground truth comes from reveal frames: the stereo depth of tissue at the first frames after the jaws open and withdraw, compared with what each method predicted for that region just before release.

Controlled synthetic data. We will render videos from finite-element liver grasps of the kind used in the Neural-Augmented Kelvinlet benchmark. That benchmark has 20,800 solutions, including nonlinear Mooney-Rivlin material. The renders give exact displacement ground truth under the jaws and test whether a linear-elastic prior misleads us on nonlinear tissue.

Regression check on visible tissue. We run on EndoNeRF and SCARED, where tool kinematics are absent and tool pose must be estimated by Instrument-Splatting render-and-compare. The aim is to confirm that visible-pixel quality holds near EndoGaussian's 37.849 dB on EndoNeRF and near LGS's 27.05 dB on SCARED LGS.

Repeated grasps. SuPer reports that a static map failed after three grasps. We will test whether contact anchoring keeps reveal error flat over five or more consecutive grasps of the same point.

Decisive checks. The proposal succeeds only if all of the following hold:

  • Tissue skating drops substantially relative to masked EndoGaussian.
  • Reveal-frame depth error in the occluded ring beats both the position-based-dynamics method and the hard-attachment ablation.
  • The world-clamp ablation is clearly worse on lift trajectories.

The proposal fails if any of these happens:

  • Error falls only inside the jaw patch, which would mean gluing suffices.
  • The position-based-dynamics baseline matches us.
  • Grasp-state errors from the soft contact estimate cause visible regressions on sequences without contact.