Weighing Hidden Interiors from How Rigid Objects Tumble

Methods that rebuild 3D objects from photos and video recover only the outer skin. The interior is treated as empty or, when a physics simulator needs mass, filled with uniform material. The one existing route to the true interior watches the object deform on impact and inverts that deformation. It fails exactly where most everyday objects live: stiff, rigid things like mugs, tools, toys and sealed boxes, which do not visibly deform. We propose a different source of evidence. A rigid object's motion when it is tossed, tipped or rolled depends on three internal quantities: where its center of mass sits and how its mass is spread around that center (its inertia tensor). A hollow-handled hammer, a mug with a weighted base and a box with a battery in one corner all tumble differently even though their surfaces look the same.

The core proposal is to recover these mass moments from casual dynamic video of an object being handled. The hand and other motion that static-scene methods throw away become the probe. We then use the moments as hard constraints on a volumetric interior density field, and report which parts of the interior the video actually pins down and which it cannot. The claim is deliberately modest and testable. Rigid motion identifies the first and second mass moments and nothing more. We build the pipeline that extracts them from in-the-wild video, and we show that a moment-constrained interior beats both uniform filling and appearance-only interior prediction on downstream physics. We also build an uncertainty estimate that honestly says "unknown" for interior structure the moments cannot see.

Where This Sits

Two strands of prior work motivate the proposal. On the surface-to-volume side, PhysGaussian fills reconstructed Gaussian splats with simulation particles using their opacity field. Fracture-GS samples the interior through a signed distance field, and DiffWind fills it with an octree of voxels. All three assume a uniform solid interior and never check that assumption against observed motion. Feed-forward predictors such as VoMP and AdaVoMP predict spatially varying density and stiffness from appearance. They work on static assets, and they cannot know that an opaque shell hides a cavity or a lead weight. We reuse such a predictor as a prior and correct it with dynamic evidence.

The closest work is Structure from Collision. It backpropagates through a differentiable particle-based continuum simulator (the Material Point Method) to recover internal cavities from impact deformation, building on PAC-NeRF's coupling of that simulator with a radiance field. Its stated limitation is our starting point: it needs visible deformation and fails on stiff objects. It also assumes a known homogeneous material and has been evaluated only on synthetic drops onto plates. MASIV learns constitutive laws from multi-view video of deforming objects, again relying on surface deformation to infer interior motion. We replace deformation with rigid-body rotation and translation, which stiff objects provide in abundance, and we accept that the recoverable information is far coarser: ten numbers, not a full field.

On the dynamic-scene side, distractor-removal methods such as RobustSplat, RobustNeRF and SpotLessSplats treat moving content as noise to be masked out. Decomposition methods such as Split4D separate moving objects from the background but keep them as hollow surfaces that move without any physics. We supply the method in the other direction. Split4D-style decomposition isolates the handled object, and robust masking removes the hand. The object's motion is then kept and interpreted physically rather than discarded.

Core Claim

For a rigid body in free flight, the center of mass follows a gravity parabola. Angular momentum about the center of mass is also conserved in the world frame. Write $R(t)$ for the object's tracked orientation and $\omega(t)$ for its body-frame angular velocity, both obtained from a rigidly constrained 4D reconstruction. Write $I$ for the unknown body-frame inertia tensor about the center of mass. Conservation then requires

$$R(t)\, I\, \omega(t) = L \quad \text{for all } t \text{ in a free-flight segment},$$

where $L$ is a constant vector. This system is linear in $I$ and $L$ and fixes $I$ up to a global scale, so its principal axes and the ratios between principal moments are identifiable. The center of mass $c$ is the body-frame point whose world trajectory $R(t)c + T(t)$ best fits a parabola with known gravitational acceleration, where $T(t)$ is the tracked translation. Contact segments add further constraints. In toppling and rolling, the object tips when the center of mass passes the support edge, and rolling speed depends on the inertia about the contact axis. Total mass stays unidentifiable without a force measurement, so we report density only up to a scale.

Given the shell domain $\Omega$ from reconstruction, a prior density $\rho_0$ (uniform, or predicted by AdaVoMP), and moment targets $(c, \hat I)$, we seek interiors that reproduce the observed motion. The minimum-divergence correction of the prior that matches these linear and quadratic moments has a closed form: an exponential tilting $\rho(x) \propto \rho_0(x)\exp(a^\top x + x^\top B x)$ restricted to $\Omega$, where the vector $a$ and symmetric matrix $B$ are chosen so the moments match. This result is a feature, not a bug. It states exactly how much structure the evidence licenses: a smooth shift of mass, not a sharp cavity. To consider sharp hypotheses we sample structured interiors from a generator (parametric cavities and inserts, or amodal part completions in the style of HoloPart). We weight each sample by how well it reproduces the moments, which yields a posterior over interiors whose spread is our uncertainty estimate.

Research Program

The first component is a dynamic capture-to-trajectory pipeline. We reconstruct the scene with a 4D Gaussian representation in the style of 4D Gaussian Splatting. We segment the handled object natively as in Split4D, then replace its free-form deformation field with a single rigid transform per frame. This change removes the jitter and inflation that kinematic warps produce. The hand is masked with robust losses while it is in contact. We segment the trajectory automatically into hand-contact, free-flight and ground-contact phases, using the residual of a rigid ballistic fit as the switch.

The second component is moment identification. We solve the linear angular-momentum system and the parabola fit by least squares. Then we refine all moments jointly by differentiating through a rigid-body simulator with ground contact over the contact phases. Physical validity, meaning a positive-definite inertia tensor that satisfies the triangle inequalities between principal moments and a center of mass inside the convex hull, is enforced by parameterizing the inertia through a Cholesky factor of the second-moment matrix.

The third component is interior inference: the exponential-tilting projection plus generator-sampled hypotheses described above. It produces a mean density field and a per-voxel identifiability map.

The analysis question running through all three is identifiability. Which interiors are distinguishable from their uniform-filled counterpart, how many free-flight frames that takes, and how errors in the tracked rotation propagate into inertia ratios.

Baselines

The naive reference point is uniform solid filling of the reconstructed shell, as done by PhysGaussian, Fracture-GS and DiffWind. It puts the center of mass at the volume centroid. We run it on exactly the same reconstructed shell our method uses, so any difference comes from the interior and not from the surface. Structure from Collision reports this uniform-interior assumption at a future-state prediction PSNR of 23.44 against 26.60 for its inversion on deformable objects, with internal-particle Chamfer Distance of 0.720 for a static NeRF baseline against 0.165 for SfC-NeRF. Those numbers come from its deformable regime, and there is no reported result found on Lacuna for rigid objects. Beating uniform fill means lower center-of-mass error and inertia error on every object whose interior is asymmetric, and a statistical tie on symmetric controls.

The strongest existing method on the interior question is Structure from Collision itself. We run it on the same rigid-object videos and give it the same training budget and multi-view input. Its material stiffness is swept upward toward rigid, because its own paper predicts collapse there. Beating it means lower moment error at high stiffness. We also report where the two methods cross over as stiffness decreases, which we expect to be informative rather than embarrassing.

The appearance-only competitor is AdaVoMP's predicted density field, used without any dynamics. Its authors report about 25% lower Young's-modulus error than VoMP on GVT-HARD, but there is no reported result found on Lacuna for center-of-mass or inertia accuracy. Beating it means our moment-constrained correction of its own prediction lowers moment error, and our full pipeline uses AdaVoMP as the prior so this comparison isolates the dynamic evidence.

Two ablations isolate the key ingredients. The first keeps the dynamics but drops the volumetric step, reporting only the ten fitted numbers. It tests whether the interior field adds anything for downstream tasks. The second keeps everything but uses the free-form 4D deformation field instead of the per-frame rigid transform. It tests whether physically constrained tracking is what makes the inertia identifiable.

Why not just…?

Why not just fill the shell uniformly, as PhysGaussian does? For a homogeneous object, uniform filling is correct, and our method should match it rather than hallucinate structure. The failure appears whenever mass is unevenly distributed. Structure from Collision's 23.44 versus 26.60 PSNR gap shows that interior errors visibly corrupt predicted motion even for deformable objects. Our synthetic suite includes symmetric and asymmetric interiors precisely so that this baseline wins where it should and loses where it should.

Why not just predict density from appearance with VoMP or AdaVoMP? These models see only the outside and are trained on assets labeled by a vision-language model, so they cannot detect a lead weight behind an opaque painted shell. This alternative is a real threat on everyday objects whose material is visible from the outside. The prior-versus-corrected comparison settles how often dynamics actually change the answer. If the correction is negligible on natural objects, that is a publishable negative result about where dynamic probing is worth the effort.

Why not just fit the ten inertial parameters and skip the volume? For predicting rigid motion alone, this is nearly sufficient, and we say so. Rigid dynamics depend on nothing else, so the first ablation should match the full method on future-motion prediction. The volumetric step earns its place only on tasks that depend on more than the moments: cutting, fracture initialization in Fracture-GS, and grasp-stability checks that integrate density near the grasp. Equally important is reporting which interior regions remain unknown. If the field gives no gain on those tasks, we will report the identification pipeline as the contribution.

Why not just run Structure from Collision with a high stiffness? Its own analysis states that stiff objects produce no usable gradients for interior density, and stiff continuum simulation is numerically expensive. The stiffness sweep in the Baselines section tests this directly rather than asserting it.

Proposed Experiments

The synthetic benchmark extends the Structure from Collision dataset's 115 shapes with cavities, and adds GVT objects with inserted dense or empty regions. All are simulated as rigid bodies being tossed, toppled and rolled, then rendered from multiple views with a synthetic hand present during contact. The metrics are center-of-mass error normalized by object diameter, relative error in the principal moments, angular error of the principal axes, internal Chamfer Distance (Structure from Collision's metric) for the posterior mean interior, and future-state PSNR after a held-out push.

The real benchmark uses 3D-printed shells with hidden weights at known positions, together with measured center of mass from a balance test. These objects are tossed and tipped by hand on a multi-view rig and captured casually. A wind-free subset of the WD-Objects capture protocol informs camera setup. A monocular subset probes the depth ambiguity that MASIV flags as open.

Three checks would decide the proposal. On objects whose cavity is centered and rotationally symmetric, the moments match a uniform solid. The method must therefore output an interior posterior indistinguishable from uniform, with high uncertainty, rather than invent a cavity; hallucination there would falsify the calibration claim. On asymmetric interiors, center-of-mass error must fall well below the uniform-fill error as free-flight frames increase. That error should decrease roughly with the inverse square root of the number of frames, because the parabola fit averages independent tracking noise. Finally, on the stiffness sweep, our moment error should stay flat while Structure from Collision degrades. If both fail at high stiffness, the rigid tracking and not the physics is the bottleneck, and that becomes the next problem to fix.