Recovering Hidden Stiff Cores from Video with Spline-Bounded Material Zones

Many soft objects are not made of one material. A plush toy has a hard plastic skeleton inside soft stuffing. A shoe is soft foam glued to a stiff sole. Current methods that learn a "digital twin" from video, a simulation whose stiffness is fitted until it reproduces observed deformation, describe stiffness as a smooth function over space. Such functions blur the sharp jump between materials into a gradual ramp. This note proposes a different description of the inside of the object: a few closed spline surfaces, each controlled by a small set of points, divide it into zones, and each zone has one constant stiffness and damping. Video then supervises both the zone stiffnesses and the positions of the spline control points through a differentiable simulator.

The claim is not only that the resulting boundaries look sharper. The claim is that a boundary described by tens of control points is a strong enough prior to make an ill-posed problem solvable: locating an interior interface that the cameras never see, from surface motion alone. A smooth neural field has thousands of degrees of freedom and can explain the same surface motion with a diffuse stiffness cloud. A parametric zone model has to commit to an inclusion of a particular shape. It should therefore generalize better to new pokes and pulls, especially when only a few interactions are observed. That is the decisive test.

What exists and what is missing

The material side supplies the pipeline and the evidence for the gap. PhysTwin fits a spring-mass network and Gaussian-splat appearance to multi-view RGBD video. It uses evolutionary search followed by gradient descent, supervised by CoTracker3 point tracks. Its per-spring parameters have no spatial structure. NeuSpring replaces those independent parameters with a tri-plane neural field queried at spring midpoints. It reports 20% better reconstruction and 25% better future-prediction Chamfer distance than PhysTwin. The improvement rests on the assumption that "nearby parts of an object typically share similar material characteristics," which is exactly the assumption that fails at a material interface. Virtual Elastic Objects fits per-particle elastic (Lamé) parameters with a quasi-static simulator and reaches 4.4 mm average error on PLUSH, but the per-particle fields are noisy. PhysDreamer uses a neural Young's-modulus field distilled from video diffusion and needs manual segmentation. Taken together, these works motivate the proposal: the structure of the stiffness field matters, and neither unstructured nor smooth structure matches piecewise-homogeneous objects.

Forward-simulation work constrains the design. Sparse, Geometry- and Material-Aware Bases and Subspace Mixed Finite Elements both show that letting smooth functions straddle a stiff-to-soft interface causes "locking," where the stiff part freezes the soft part. A smeared interface is therefore a source of wrong dynamics, not just a cosmetic flaw. Both methods are forward-only and assume the interface is already known on a tetrahedral mesh. We aim to infer it.

The boundary side supplies the method. SplineDist represents cells as star-shaped spline contours around a center. BPNet fits Bézier patches to point clouds with a differentiable reconstruction loss. ExtrudeNet optimizes rational Bézier loops through a differentiable signed-distance conversion with no ground-truth supervision. None of these methods attaches physics to the curves. Spline Deformation Field uses splines along time to describe motion and has no material parameters. Its spatial field is still an implicit network. MIND builds multi-region "material interfaces" only in a geometric sense. As far as the file shows, no work optimizes spline control points to divide an object into zones of constant material inside a differentiable physics fit.

Core Claim

Represent the inside of a deformable object as $K$ nested or disjoint closed spline surfaces plus a background zone, and give each zone constant stiffness and damping. Couple these zones to a spring-mass simulator in a way that is exact in the sharp limit. Under that setup, inferring material from video will:

  1. recover hidden interfaces more accurately than smooth or per-spring fields;
  2. predict responses to unseen interactions better, with the margin growing as training interactions become fewer;
  3. avoid the locking-like over-stiffening that smeared interfaces cause.

Research Program

Interface surfaces. Each zone boundary is a star-shaped surface around a learnable center $\mathbf{c}_k$. Its radius in direction $(\theta,\phi)$ is a tensor-product B-spline $R_k(\theta,\phi)$ defined by a grid of roughly 6×8 control radii. This is the 3D extension of SplineDist's contours. It makes the inside test analytic, so that $\mathbf{x}$ is inside zone $k$ when $\|\mathbf{x}-\mathbf{c}_k\| < R_k(\theta,\phi)$. It also gives smooth normals for free. Star-shaped surfaces cannot represent every geometry; for example, a branching armature does not fit. We handle elongated parts by allowing several overlapping zones that share a material label. We report failures on non-star-shaped cores honestly instead of claiming full topological generality.

Coupling to springs. The key technical step is how a moving surface changes spring stiffness without remeshing. A spring is a one-dimensional element. When it crosses an interface, its sub-segments act in series. Let $f_{s,k}$ be the fraction of spring $s$'s length lying in zone $k$, and let $k_k$ be zone $k$'s stiffness per unit length. The effective stiffness is then

$$\frac{1}{k_s^{\text{eff}}} = \sum_{k} \frac{f_{s,k}}{k_k}.$$

This is exact for a piecewise-uniform rod. We compute $f_{s,k}$ by sampling points along the spring and passing the signed inside test through a sigmoid whose sharpness we raise during optimization. Gradients reach the control points through $f_{s,k}$. At full sharpness the model becomes a true step-function material, and springs that cross the interface get the physically correct series stiffness rather than an average. Damping is combined the same way. As an ablation, we replace the series rule with a plain arithmetic blend. The arithmetic blend lets the stiff zone dominate any spring that crosses into it, which is the spring-network analogue of locking.

Fitting. We reuse PhysTwin's capture pipeline, shape prior, spring topology, and CoTracker3 supervision unchanged, and swap only the stiffness model. Optimization alternates between the zone stiffnesses (log-parameterized) and the control points. The number of zones $K$ is chosen on held-out interactions within each object's training split. To avoid bad local minima, each zone center is initialized from regions where tracked strain is unusually low.

Identifiability analysis. From surface observations alone, a small, very stiff core and a larger, moderately stiff core may produce the same motion. We will measure this directly. On synthetic objects we sweep the stiffness contrast and the depth of the inclusion, and record how interface error grows. The goal is to map where interior interfaces are recoverable from video and where they are not.

Baselines

The naive reference point is a single homogeneous stiffness per object, fitted with the same simulator. It shows how much any heterogeneity buys on each object.

The strongest existing method is NeuSpring's tri-plane stiffness field. For future prediction on the PhysTwin dataset, the file reports only that NeuSpring improves Chamfer distance by 25% over PhysTwin, and by 20% for reconstruction (NeuSpring). No reported absolute Chamfer value was found on Lacuna for either method. We will therefore rerun both under identical conditions: the same spring topology, the same tracks, the same 70/30 train/test split, the same number of optimizer steps, and an equal hyperparameter search budget per method. Beating it means lower future-prediction Chamfer on the PhysTwin test split, averaged over its 22 scenarios. We expect the largest gains on stuffed animals and packages with rigid contents, and no loss on cloth, where a single zone should be selected.

PhysTwin's per-spring parameters are the unstructured baseline. On PLUSH, Virtual Elastic Objects' 4.4 mm average error is the reported reference. We will run our zone model inside a comparable quasi-static setting there. Because the simulators differ, we present this comparison as secondary.

The ablation that isolates the key ingredient is a K-zone neural label field. It has the same zones, the same per-zone constants, the same series coupling, and the same sharpness annealing. The only difference is that zone membership comes from a coordinate network with a softmax over $K$ labels instead of spline surfaces. If this ablation matches the spline model, the gain comes from piecewise homogeneity, not from parametric boundaries. A second ablation replaces the series rule with arithmetic blending.

Why not just…?

Why not just regularize NeuSpring's tri-plane with total variation so it becomes piecewise constant? An edge-preserving penalty does push smooth fields toward sharp steps. It still leaves thousands of free values in the interior, where the data says almost nothing, and it cannot express the idea "there is one compact inclusion." We include a total-variation tri-plane at a tuned penalty weight as an additional baseline. The prediction is that it matches us when many interactions are observed and falls behind when only one or two interactions per object are used for training. If it does not fall behind, the low-dimensionality argument fails.

Why not just fit NeuSpring and then cluster or threshold its stiffness into zones? Clustering after the fact inherits the blurred interface. It places the boundary somewhere inside the transition ramp, and the cluster stiffnesses were fitted under the wrong spatial layout. We test this by clustering NeuSpring's field with k-means into the same $K$ zones, refitting only the zone constants, and comparing interface error and future prediction against joint optimization of boundaries and stiffness.

Why not just use a K-label neural field, since it already gives piecewise-homogeneous zones? This is the real threat. It is why that model is the central ablation rather than a strawman. Our case for splines rests on two predictions. First, a few dozen control points recover hidden interfaces more reliably than a free-form label network under sparse supervision. Second, the label network will tend to carve zones that follow the visible surface rather than the hidden core. If the label field ties us across the contrast-and-depth sweep, the honest conclusion is that piecewise homogeneity is what matters. The spline contribution would then shrink to compactness and CAD-friendly export.

Proposed Experiments

Real data cannot reveal hidden stiffness maps, so the main experiment is a synthetic benchmark with known interiors. We take the 14 asset shapes used in the Spring-Gaus synthetic setup and embed one to three stiff inclusions of controlled shape, depth, and stiffness contrast (from 3× to 100×). The ground-truth videos are rendered with a Material Point Method simulator, not with our spring-mass model. This avoids testing a simulator against its own output. Metrics are:

  • interface Chamfer and Hausdorff distance between recovered and true zone surfaces;
  • per-zone log-stiffness error;
  • future-state Chamfer on held-out interactions;
  • an over-stiffening measure: mean strain in the soft region near the interface, predicted versus true.

The decisive checks are two curves comparing the spline model against the label-field ablation and the total-variation tri-plane, as the number of training interactions drops from many to one. The first curve is interface error versus inclusion depth. The second is future-prediction error versus number of interactions.

On real data we evaluate future-prediction Chamfer and tracking error on the PhysTwin dataset against NeuSpring and PhysTwin, as described in the Baselines section. We also give a qualitative check: for plush toys whose internal structure can be confirmed by cutting one open or by documentation, do the recovered zones sit where the armature is? A secondary run on PLUSH compares against the 4.4 mm reported by Virtual Elastic Objects.

The proposal succeeds if three things hold: the spline model beats NeuSpring on PhysTwin future prediction, it beats the label-field ablation on synthetic interface recovery in the low-data regime, and the series coupling measurably reduces over-stiffening relative to arithmetic blending. If only the first holds, the result is that piecewise homogeneity helps. That is useful, but it is a weaker claim than the one made here.