Occluders as Evidence: Non-Penetration Constraints for Hidden Surfaces in Dynamic Reconstruction

When a camera films a hand pressing into a towel, the patch of towel under the fingers is never seen. Current methods that rebuild a moving scene in 3D fill that patch in one of two ways. They either stretch nearby visible geometry smoothly into the gap, or they ask an image generator to invent it. Neither method knows that the hand is solid. So the reconstructed towel often passes straight through the fingers, or it hangs flat behind them where it should dent. We propose to treat the occluder's own reconstructed volume as a measurement of the hidden surface. Two solids cannot occupy the same space, and where they visibly meet they touch. Together these facts bound where an unseen deforming surface can be at every moment. Because the occluder moves, these bounds pile up over time on the same underlying surface point.

This matters because hidden regions are exactly where dynamic reconstruction breaks. They produce the floaters, smears and depth jumps that appear when such a scene is viewed from a new angle. Physical-plausibility methods for hand-object interaction already penalize interpenetration, but only for visible, rigid, template-based objects. Dynamic reconstruction methods already model non-rigid deformation, but they have no notion of contact. Joining the two in the regime where one deforming body hides another is, as far as the literature we have access to shows, untested.

Core Claim

In monocular or sparse-view capture of interacting deformable bodies, we claim non-penetration is most useful not as a cleanup term on visible geometry but as the main geometric evidence for surfaces that are disoccluded, meaning surfaces that are hidden for part of the sequence and later revealed or viewed from a new angle. The mechanism is a two-sided constraint we call a "contact bracket," applied per body and per frame on top of a canonical deformation model. A canonical deformation model keeps one reference shape per body plus a learned function that warps it to each time step. The bracket has two halves. A repulsive bound applies everywhere and stops a hidden point from entering another body. An attractive bound applies only where the visible images show contact, and pins the hidden patch to the occluder's surface. Contact is inferred from the image itself: at an occlusion boundary, the occluder's depth and the occluded surface's depth agree. No CAD model of either body is needed. The testable prediction is that this bracket cuts geometric error in hidden regions much more than any regularizer that only asks for smooth extrapolation. It should do so while barely changing error in visible regions and without the floating that repulsion alone causes.

Relation to Prior Work

The physical-plausibility work supplies our loss terms, but it assumes known rigid objects. Keep It SMPL penalizes intersecting capsules on a parametric body. Grasping Field penalizes points where two signed distance fields are both negative; a signed distance field gives, for each point in space, its distance to a surface, with negative values meaning inside. CPF balances attractive and repulsive virtual springs, but it requires a pre-scanned object and works on single frames. ContactOpt shows that forbidding all penetration gives unstable gradients, and it deliberately allows about 2 mm of overlap to mimic soft tissue. We reuse that tolerance directly. The overlap test from real-time two-hand reconstruction computes intersections between 3D Gaussians in closed form, and we use it to keep the check cheap. The difference from all of these is threefold. Both bodies in our setting are reconstructed and deforming. Contact is inferred from occlusion boundaries rather than predicted from a template. The constraint's main job is to supply evidence where cameras have none.

The dynamic-reconstruction and correspondence work motivates the gap. Deformable 3D Gaussians and D-NeRF warp a canonical scene over time but enforce no boundaries between parts. 4DTAM regularizes unobserved zones with an as-rigid-as-possible loss, which asks each small patch to move like a rigid piece. OAR makes occluded points follow their visible neighbours through locally linear reconstruction. Both are extrapolation priors. A pressed towel is precisely the case where extrapolation is wrong: the smooth continuation of the visible cloth runs through the finger. DG-Mesh gives time-consistent meshes but no inter-body constraint. R5DGS groups Gaussians into rigid instances, which does not transfer to deforming bodies, and does not stop those groups from interpenetrating.

The closest multi-direction work is ViTaM-D. It removes hand-object interpenetration with deformable objects, but it gets its contact signal from a tactile glove, and it is evaluated in simulation. We aim to recover that signal from vision alone.

The generative-disocclusion work is the competing philosophy. FreeOrbit4D completes unseen object backsides with a multi-view diffusion model, which builds an image by repeatedly denoising random noise. It then snaps the completed object into the scene with no contact physics. Beyond the Visible handles reveals with articulated proxy graphs, and its authors state that it fails for cloth and other continuous deformation. We treat generation as complementary: it can paint appearance on never-seen regions, but the bracket should decide where that geometry sits.

Research Program

The representation follows 4DTAM. Each body is a set of surface-aligned 2D Gaussians in a canonical space, deformed by its own time-conditioned warp network. Bodies are separated either with a video segmenter or natively in the style of Split4D. We track, for each canonical Gaussian, whether it has been visible in any frame so far. Hidden Gaussians are those never visible at the current time or across the sequence. For a Gaussian of body $A$ at centre $x_i(t)$, let $d_B(x,t)$ be the approximate signed distance to body $B$ at time $t$ (negative inside), computed from Gaussian overlaps. Let $\varepsilon \approx 2$ mm be the soft-tissue tolerance, $w_i$ a weight that is larger for hidden Gaussians, and $c_i(t)\in[0,1]$ the inferred contact indicator. The bracket is

$$ \mathcal{L}_{\text{bracket}} = \sum_{t}\sum_{i} w_i \left[ \max\!\big(0,\,-d_B(x_i(t),t)-\varepsilon\big)^2 + c_i(t)\, d_B(x_i(t),t)^2 \right]. $$

The first term forbids penetration deeper than the tolerance. The second pulls a Gaussian onto the other body's surface only where contact is evidenced. The contact indicator is set at occlusion boundaries: $c_i(t)$ is positive when the rendered depth of the occluder and of the visible fringe of the occluded body agree within a threshold. That positive value is then spread to hidden Gaussians of body $A$ lying beneath the occluder's surface near that boundary. Because every term is expressed on canonical points, a hidden point collects constraints from every frame in which a moving occluder sweeps near it. This time-accumulation is what makes a one-sided bound informative.

We will analyze three questions. The first is how much hidden-region error each half of the bracket removes on its own. The second is whether the improvement grows with how much the occluder moves, as the accumulation argument predicts. The third is whether a diffusion inpainter conditioned on the bracketed geometry gives fewer depth jumps than one that sets geometry and appearance jointly.

Baselines

The naive reference is Deformable 3D Gaussians (Yang et al.) run as-is on the same input, with no treatment of hidden regions. It shows the error the problem starts from. The strongest geometric competitor is 4DTAM (4DTAM), which is built for non-rigid RGB-D reconstruction and already regularizes unobserved regions. Its reported absolute trajectory error on the Sim4D curtain sequence is 0.25 cm, against 6.10 cm for SurfelWarp. We will rerun it on our splits with its released hyperparameters and our backbone, so that the only difference is the hidden-region term. Beating it means lower geometric error on hidden regions with a matched budget, and visible-region error and trajectory error no worse than its own on the same run.

The locality prior from OAR is plugged into the same backbone as a second extrapolation baseline. For generative completion we run FreeOrbit4D on the monocular inputs and measure both its geometry and its interpenetration. For the physical-plausibility side, we apply a ContactOpt-style post-hoc refinement (ContactOpt) to the output of the naive baseline. On HO-3D, ContactOpt reports reducing hand joint error from 57.7 mm to 48.1 mm, and it is the most direct test of whether in-the-loop constraints beat after-the-fact correction. CPF reports reducing penetration depth by 3.71 mm and intersection volume by 9.34 cm³ on the First-Person Hand Action benchmark, and serves as a reference for penetration metrics.

Three ablations isolate the key ingredient. The first is the repulsive term alone. The second is a uniform penalty with $w_i$ constant, the "just add a collision loss" version. The third is the bracket with contact inferred from ground truth rather than from occlusion boundaries, which bounds how much the contact detector costs us. All methods get the same frames, masks, iteration count and per-method tuning grid of equal size on a held-out validation split. We found no reported result on Lacuna for any method's geometric error restricted to hidden regions, so these numbers will be produced here.

Why not just…?

Why not just use the smoothness regularizers that already exist for unobserved regions? The as-rigid-as-possible loss in 4DTAM and the locally linear reconstruction prior in OAR are cheap, and for surfaces that are merely hidden, not pressed, they may be enough. This is a real threat. Our prediction is specific: they should match the bracket when the occluder hovers, and lose when it presses or grips, because the smooth continuation then runs through the occluder. We will stratify results by measured contact, and a null result in the contact stratum would falsify the core claim.

Why not just add a collision penalty to an existing deformable model and stop there? This is the obvious combination, and the uniform-weight ablation is exactly it. If a uniform penalty recovers most of the gain, we will report that plainly. The claim is then reduced to showing where the benefit of collision losses actually comes from, and the method contribution shrinks to the contact term. We expect repulsion alone to push hidden patches back into free space and cause floating, which is the trade-off noted for CPF and ContactOpt. The attractive term exists only to counter this, and it is measured separately.

Why not just let a generative model complete the hidden geometry? FreeOrbit4D already completes unseen regions with a multi-view diffusion model. That model has never seen the occluder's volume, so nothing stops the completed geometry from intersecting it. We measure its intersection volume and hidden-region error directly. We also test the hybrid, bracket geometry with generative appearance, to show whether the two are complementary rather than competing.

Why not just post-process with ContactOpt? Post-hoc refinement fixes a single frame's pose against a rigid template. It cannot move a deforming surface's canonical points consistently across time. The comparison above settles whether that matters.

Proposed Experiments

Hidden geometry needs ground truth that the input camera lacks. We get it in two ways. On DexYCB, which has eight synchronized RGB-D cameras, we reconstruct from one camera and score geometry against depth from the held-out cameras. We score only on surface points that the input camera never saw. We report Chamfer distance there, and penetration depth and intersection volume between hand and object. DexYCB reports 27.5 mm hand error for depth-based estimation and 52.2 mm for RGB-only, and these give a sense of the achievable scale. Its objects are rigid but the hand is not, so both directions of occlusion are exercised.

For deformable occluded bodies, we build synthetic composites. Animated occluders from Sim4D and DT4D press into or pass over cloth and animal meshes, and we render them from a single moving RGB-D camera. Full ground-truth surfaces are then available at every frame, including contact ground truth for the oracle ablation. We also report the standard Sim4D trajectory and depth metrics, and HO-3D penetration depth and intersection volume, to confirm that visible-region quality and tracking do not regress. For viewing from new angles, we render at wide baselines on the synthetic composites and report PSNR and LPIPS inside disoccluded masks only.

The decisive checks are three. First, in the contact stratum, the bracket must beat 4DTAM-style regularization on hidden-region Chamfer distance while keeping visible-region error within noise. Second, hidden-region gains must grow with occluder motion, since the accumulation argument says a stationary occluder gives only one bound. Third, the full bracket must reduce floating, measured as the gap between the hidden surface and the occluder at true contacts, relative to repulsion alone. If the first check fails, the idea is wrong. If only the third fails, the contact detector is the weak point, and the oracle ablation will say how much.