When a person walks, a planted foot stops moving for a fraction of a second, and during that time it is, physically, part of the static world. Current systems ignore this. Methods that reconstruct a moving person and a static room from one video either label every body point as "moving", or treat contact as something a separate single-image network guesses frame by frame. The result is foot sliding (feet gliding across the floor while supposedly planted). Meanwhile the ground under the foot, which is hidden by the foot itself, is reconstructed worst of all. We propose to invert the relationship. Contact should be inferred from the reconstruction: a body point that is momentarily motionless in world space and sits on the scene surface is in contact. Each such contact then becomes a direct measurement of the hidden support surface. A single joint optimization over a fast signed distance field (a function giving each 3D point its distance to the nearest surface, negative inside objects) should remove sliding and fill in occluded support geometry on stairs, furniture and uneven ground, where flat-floor fixes cannot help.
The opportunity is that three unsolved problems share one root. Foot sliding persists because the moving-person model has no boundary condition tying it to the static scene. Methods that split a video into static and moving parts break down exactly at contact patches, where it is unclear where the floor ends and the foot begins. Neural surface fields are slow partly because they spend equal effort everywhere, although physical plausibility only requires accuracy in a thin band where the body touches the world. A method that makes contact, the static/moving boundary, and the region of costly geometric accuracy the same latent object addresses all three.
Core Claim
Treating stance contact as a jointly inferred latent variable should beat attaching a contact loss to an existing human-scene reconstructor. The variable is scored by world-space stillness and scene-surface proximity, and smoothed in time by a touchdown–stance–liftoff model. The advantage should appear in two measurable places: less foot sliding on non-flat supports, and lower geometric error of the support surface directly under the feet. We expect parity, not gains, on flat floors. There a flat-ground zero-velocity heuristic is already close to sufficient, and we treat that parity as a sanity check, not a result.
Relation to Prior Work
The closest system is HSR. It models a monocular-video human and a static indoor scene as neural signed distance fields in one coordinate frame and penalizes their volume overlap. The penalty stops feet from sinking but does nothing about lateral gliding, and HSR has no contact state. Kinematic footskate reducers take the opposite approach. Reducing Footskate with Ground Contact Constraints predicts toe and heel contacts from 2D keypoints and imposes zero velocity, reporting contact mAP 0.9292 on Human3.6M, but it assumes a flat ground plane and reconstructs no scene. SHARE anchors the root trajectory to a metric background point map, but it optimizes only global translation and explicitly leaves floor penetration and sliding open. These three works motivate the proposal: each solves one half of the coupling.
Two works supply ideas we reuse. CRISP already uses the body as a probe to hallucinate occluded support structures, but it gets contacts from a vision-language detector and commits to planar cuboids refined by reinforcement learning in a simulator. We keep the probe idea, replace cuboids with a continuous field so that slopes and curved furniture are representable, and infer contact from reconstruction dynamics rather than from a detector. DECO and BSTRO give dense per-vertex contact from single images. We use DECO only as a prior, because single-image predictions flicker in time, and BSTRO itself documents that contact regions are usually self-occluded.
For speed, we reuse multiresolution hash encodings with fast second-order derivatives from NeuS2, which reports 5 minutes per static scene against 8 hours for NeuS. NeuS2 is multi-view and has no contact model. Physics-in-the-loop methods constrain the design. DiffPhy cuts foot skating on Human3.6M from 47.5% to 7.4% but needs hand-specified scene obstacles. Contact and Human Dynamics from Monocular Video takes 30–60 minutes per 2-second clip on a flat plane. Both show that full dynamics is too slow and too scene-agnostic to be the first tool here, so we stay kinematic plus geometric and leave forces out of scope.
Research Program
The representation has two parts. The scene is a static signed distance field $f$ on a hash grid with a small decoder. The person is an SMPL body (a standard parametric human mesh) whose world-space vertex positions $\mathbf{x}_v(t)$ come from per-frame pose and a global trajectory, with a canonical-space appearance field as in HSR. Initialization comes from off-the-shelf human mesh recovery and metric point maps in the style of SHARE, with the person masked out.
For each sole vertex $v$ and frame $t$ we compute a contact evidence score from the current reconstruction:
$$e_{v,t} = \exp\!\big(-|f(\mathbf{x}_v(t))|/\sigma_d\big)\,\exp\!\big(-\|\mathbf{x}_v(t{+}1)-\mathbf{x}_v(t)\|/\sigma_v\big),$$
where $\sigma_d$ and $\sigma_v$ are distance and speed scales in metres and metres per frame. This evidence, combined with DECO's per-frame probability as a prior, drives a three-state temporal model per foot region (swing, stance, and a short transition state). Forward–backward inference in that model gives soft contact posteriors $c_{v,t}$ that cannot flicker faster than the model allows.
The joint objective adds one coupled term to HSR-style photometric, mask and non-penetration losses:
$$L_{\text{contact}} = \sum_{v,t} c_{v,t}\Big(\|\mathbf{x}_v(t{+}1)-\mathbf{x}_v(t)\|^2 + \lambda\, f(\mathbf{x}_v(t))^2\Big).$$
The first part is the zero-velocity anchor. The second part writes the footprint into the scene as a zero-level sample. That sample lands exactly where the camera cannot see the floor, so it is new information, not a regularizer duplicating photometric evidence.
Optimization alternates between two steps: geometry and motion updates given the contact posteriors, and contact inference given the current geometry and motion. The alternation is EM-like. It lets a foot on a stair tread pull the tread to the right height, and the corrected tread then confirms the contact.
The efficiency component follows from the same variable. The body's swept volume is free space, and accumulated footprints mark where precise geometry matters. We therefore allocate the costly Eikonal regularization (the constraint that the field's gradient has unit length, which requires second derivatives) and the finest hash levels densely in a narrow band around the footprints and body sweep, and only sparsely elsewhere. We will measure whether this band-limited allocation reduces wall-clock time without increasing support-surface error, compared with uniform allocation at matched step count.
Baselines
The naive reference is the initialization alone. That is per-frame human mesh recovery placed in the stitched metric point map, with no contact term, which is essentially SHARE's pipeline. SHARE reports root position error (MRPE) of 0.44 m on RICH. Any method must at least preserve that root accuracy while reducing sliding.
The strongest competitor on this exact question is what we call HSR-plus-contact: HSR's joint signed-distance reconstruction with a Zou-style zero-velocity loss driven by frame-wise DECO contacts. It is precisely the "just add a loss" design a reviewer would build, and on flat floors it may match us. It will be run in our codebase on the same hash-grid backbone, the same initialization, the same number of iterations and the same GPU, with its loss weights tuned by the same grid search on the same held-out RICH sequences. That way the only difference is whether contact is a fixed detector output or a jointly inferred latent that also writes footprints into the scene. For plain HSR, the file reports 0.757 3D IoU and 15.14 cm depth error on SHSD. We will reproduce those numbers first, and then any gain in our method can be attributed to the coupling rather than to reimplementation differences. No reported runtime for HSR was found on Lacuna, so we will measure it.
We also compare against a flat-ground kinematic footskate reducer in the style of Zou et al., and against a CRISP-style planar-primitive scene with the same contact posteriors. The planar comparison isolates the value of a continuous field on non-planar supports. For reference, CRISP reports a 6.9% motion-tracking failure rate against 55.2% for raw meshes, but that is a simulator metric we do not optimize, so it is context rather than a target. The key ablations remove the footprint term ($\lambda=0$, which keeps zero velocity but gives no geometric write-back), replace the temporal model with thresholded DECO, and replace band-limited allocation with uniform allocation.
Beating HSR-plus-contact means three things on non-flat RICH and PROX subsets: a statistically clear reduction in foot-sliding rate at equal or better MPJPE, lower support-surface error under footprints, and no regression on flat Human3.6M sequences.
Why not just…?
Why not just add a zero-velocity loss with DECO contacts to HSR? This is the real threat, and it is our main baseline rather than a straw man. Its expected weakness is specific. With a fixed detector, a correctly detected stance foot on a mis-reconstructed stair tread must choose between staying still and staying on the surface, because nothing lets the contact correct the tread. Frame-wise DECO also flickers across touchdown and liftoff, which is the failure the research file identifies for single-image contact predictors. The $\lambda=0$ ablation and the thresholded-DECO ablation separate these two effects. If both ablations match the full method on non-flat sequences, the coupling claim is wrong and we will report that.
Why not just use metric point maps and inverse kinematics, as SHARE-like pipelines would? Point maps are fast and metrically scaled. But the patch under a planted foot is exactly the part no frame sees, so the point map has a hole where the constraint is needed. A point cloud also offers no signed distance to tell "above" from "inside". Contact and Human Dynamics from Monocular Video shows that purely kinematic foot fixes can imply ground forces above 9,000% of body weight. Our support-surface error under footprints, measured against RICH laser scans, directly tests whether footprint write-back fills that hole better than the point map does.
Why not just use dynamic Gaussian splatting with a static/moving split such as Split4D, then lock feet afterwards? Splatting trains fast. However, RobustSplat documents that moving content triggers spurious densification that leaves floaters, and SplatSDF notes that Gaussians give no clean zero-level set. A post-hoc foot lock also cannot move the surface it locks onto. We include a splatting variant with post-hoc locking on the same sequences, so that support-surface error and sliding reveal whether the lack of signed distance matters in practice.
Why not just fit planes, as CRISP does? On strictly planar rooms, planes may be as good as a continuous field and much cheaper. We expect this, and we will report it. The planar-primitive baseline with our own contact posteriors tells us whether the continuous field earns its cost on slopes, curved furniture and irregular outdoor ground.
Proposed Experiments
The primary benchmark is RICH, which has laser-scanned scenes, vertex-level contact labels and stationary cameras. We split its sequences by the ground-truth contact labels into floor-only and non-floor-support subsets. Metrics are foot-sliding rate (the fraction of labelled-contact frames where the sole moves faster than a threshold in world space), MPJPE, MRPE against SHARE's 0.44 m, contact F1 against DECO's reported 0.70, a new temporal-flicker count of contact transitions per second, and Chamfer error of the reconstructed scene restricted to a 10 cm band around ground-truth contact regions. PROX adds scanned indoor rooms with furniture contact. There PhySIC reports contact F1 0.51 and per-vertex scene error 227 mm, from single images; we compare against it on the same frames. SHSD checks that whole-scene IoU and depth do not fall below HSR's reported 0.757 and 15.14 cm. Human3.6M is the flat-floor parity check, where DiffPhy's reported 7.4% skating rate is the reference. EMDB is an exploratory test with a moving camera.
The decisive checks are three. First, on RICH's non-floor-support subset, the full method must beat HSR-plus-contact on sliding and on under-foot surface error, and the $\lambda=0$ ablation must fall between the two. That ordering is what shows the footprint write-back, not the zero-velocity term, does the work. Second, the temporal contact model must reduce flicker relative to thresholded DECO without lowering contact F1. Third, band-limited allocation must reach the uniform-allocation under-foot error in less wall-clock time on the same GPU. Wall-clock is reported per sequence for every method, because runtime numbers for the monocular human-scene baselines were not found on Lacuna and have to be measured here.
References
- HSR: Holistic 3D Human-Scene Reconstruction from Monocular Videos
- Reducing Footskate in Human Motion Reconstruction with Ground Contact Constraints
- SHARE: Scene-Human Aligned Reconstruction
- Contact-guided Real2Sim from Monocular Video with Planar Scene Primitives
- DECO: Dense Estimation of 3D Human-Scene Contact In The Wild
- Capturing and Inferring Dense Full-Body Human-Scene Contact
- NeuS2: Fast Learning of Neural Implicit Surfaces for Multi-view Reconstruction
- Differentiable Dynamics for Articulated 3d Human Motion Reconstruction
- Contact and Human Dynamics from Monocular Video
- RobustSplat: Decoupling Densification and Dynamics for Transient-Free 3DGS
- SplatSDF: Boosting Neural Implicit SDF via Gaussian Splatting Fusion