Footstep Odometry: Planted Feet as Camera Anchors When the Background Has No Texture

A camera filming a walking person in a blank white room cannot tell how it is moving. Visual odometry (estimating camera motion by tracking image points from frame to frame) needs textured, static background, and minimalist interiors, corridors, and dark stages often have none. Current systems that recover a person's 3D motion in world coordinates run visual odometry first and recover the body afterward. When the first stage fails, the camera error turns directly into world-space foot sliding, where a supposedly planted foot glides across the floor.

The scene still contains one reliable static object: the person's stance foot. While a foot is planted, its contact point is fixed in the world. It is visible in the image through heel and toe keypoints, and body-reconstruction networks estimate its metric depth to within roughly 0.1 m. We propose to insert these planted-foot observations into the camera's pose optimization as static landmarks, with metric depth attached. As the person walks, the feet hand off from one to the other, so the anchors chain together much like leg odometry on a walking robot. The camera can then be tracked when the background gives out, and the corrected camera removes the foot sliding that camera error had caused. In current pipelines contact information flows only from the camera to the body. This proposal sends it from the body back to the camera.

Core Claim

During each stance interval of a foot, the observed contact point is a static, metrically scaled world landmark. A pose optimizer that weights these landmarks against background features, scaled by how much background texture is available, will degrade gracefully as texture disappears. Background-only odometry, human masking, and learned human-root dead reckoning all collapse under the same conditions.

Concretely, let $T_t$ be the world-to-camera pose at frame $t$ and $p_k$ the unknown world position of the contact point during stance interval $k$. Let $u_t$ be the detected 2D heel or toe keypoint and $x_t$ the body network's metric 3D estimate of that point in camera coordinates. Each stance frame then contributes a residual:

$$ r_{t,k} = \big\| \pi(T_t p_k) - u_t \big\|^2_{\Sigma_u} + \big\| T_t p_k - x_t \big\|^2_{\Sigma_x(t)} $$

Here $\pi$ is the camera projection, and $\Sigma_u, \Sigma_x(t)$ are the keypoint and depth uncertainties. All $p_k$ are additionally constrained to a shared gravity-aligned floor plane. A gyroscope, or the gravity estimate from the body model, fixes roll and pitch. The foot residuals therefore mainly constrain translation and heading, which are exactly the quantities that drift along a featureless corridor axis, as Ishizuka and Yamashita show.

Relation to Prior Work

Three world-grounded recovery systems motivate this work, because all three treat odometry as an unaided front end.

  • HumanMM masks human pixels before running visual odometry, which cut translation error by about 50%. It then removes sliding with contact-aware trajectory refinement. With a blank background, however, masking leaves the odometry nothing to track.
  • GVHMR takes camera rotation from DPVO and locks predicted stationary joints in a post-processing step. The locking never feeds back into the camera estimate.
  • Humans as Checkerboards (HAC) uses contact-joint depth to calibrate the scale of a monocular map. Its scale is a ratio of human depth to map depth, and it is undefined when the map is empty.

We keep HAC's insight that human depth is metric. We also keep GVHMR's stationary-probability predictions. What we change is the use of contact: it becomes a geometric factor inside camera tracking rather than a post-hoc correction or a scale ratio.

The mechanism comes from Contact Inertial Odometry. That work bounds a blind drone's inertial drift with zero-velocity constraints applied at the moment it collides with a wall. We transfer the idea from the robot's own body to an observed subject. That requires handling visual, uncertain contact detection and heel-to-toe roll rather than hard impacts.

Two papers constrain the design.

  • FootMR shows that foot pose from standard in-the-wild models is inaccurate because training keypoints stop at the ankle. We therefore use explicit heel and toe keypoints, and we let the anchor point migrate from heel to toe within a stance rather than locking a rigid flat foot.
  • MultiPhys shows that hard contact enforcement raises acceleration jitter. Our constraints are soft and weighted by uncertainty, and we report jitter explicitly.

CRISP hallucinates support surfaces from contact but assumes the camera is already known. Our floor-plane factor is a minimal version of that idea, used to solve for the camera instead.

Research Program

We will build the optimizer on top of DPVO, which is the odometry used by GVHMR.

Optimizer. Background patch factors and foot-anchor factors share a sliding window. Each background factor is down-weighted by a texture-confidence score, defined as the count and spread of patches with low residuals. Contact intervals and their probabilities come from GVHMR's stationary head. Metric 3D foot points come from the body network.

Analysis. We will characterize when foot anchors are informative. With a single planted foot and known rotation, translation is observable up to the depth noise of that foot. Averaging over a stance of $n$ frames should reduce the per-interval anchor error roughly as $1/\sqrt{n}$, assuming depth errors are independent across frames. We will measure whether that independence actually holds, since errors from the body network are likely correlated over time.

Closing the loop. The refined camera is fed back to the body-recovery stage, and we measure the resulting world-space sliding.

The scope is third-person footage in which the feet are visible. Egocentric video, where feet are usually out of view, is excluded.

Baselines

The naive reference is unmodified GVHMR: DPVO on masked frames, a learned root trajectory, and post-hoc foot locking. Its reported World-MPJPE on EMDB is 274.9 mm, compared with 335.3 mm for WHAM (GVHMR paper). These numbers come from normal-texture footage. We found no reported result on Lacuna for camera trajectory error or foot-skating rate on EMDB for either method, so we will measure both ourselves.

The strongest existing competitor combines the HumanMM-style recipe (human-masked odometry plus contact-based trajectory refinement) with HAC scale calibration. HAC reports a 50% reduction in global motion error over local-to-global methods. Lacuna gives no absolute EMDB value for HAC.

Fairness. All baselines receive the same frames, the same body network, the same contact predictor, the same gyroscope signal where it is used, and the same window length. Hyperparameters for each method are tuned on the same held-out EMDB sequences with an equal search budget.

Ablations isolating the key ingredient. - Foot keypoints are added to DPVO as ordinary static patches during predicted stance, with no metric depth term. This tests whether metric depth matters or whether simply un-masking the stance foot is enough. - The foot factors are kept but the texture-adaptive weighting is removed.

What counts as beating the baselines. We must lower camera translation error and foot-skating rate across the low-texture range of the degradation curve described below, and stay within noise of the baselines on full-texture footage.

Why not just…?

Why not just use structural odometry like PLP-VIO? PLP-VIO adds lines and planes. It reduced translation error on EuRoC MH_02 from 15.3 cm to 10.9 cm. It needs detectable wall–floor edges, however, and it does not separate human contours from static structure. We include it as a baseline on the real featureless captures. If it succeeds there, our contribution narrows to rooms that lack clear structural lines.

Why not just fuse an IMU? Commercial headsets fuse inertial sensors with dedicated hardware, yet XR Reality Check still finds error spikes of up to 101% at blank walls. Double-integrated accelerometer drift is unbounded without external anchors. We do use the gyroscope, because rotation is cheap to get. We will report gyroscope-plus-accelerometer VIO as a baseline so the reader can see whether translation is what fails.

Why not just derive the camera from the learned human root trajectory plus a gyroscope? This is the real threat. GVHMR-style models predict the person's world motion from body motion alone, so the camera could be obtained by composing that trajectory with the body-in-camera estimate, with no texture needed. Learned root velocity is a form of statistical dead reckoning, though. The 274.9 versus 335.3 mm gap between GVHMR and WHAM shows how much such priors vary. Foot anchors are hard geometric constraints. We run exactly this alternative as a baseline. If it matches us on the degradation curve and on the real captures, the geometric factor adds nothing and we will report that.

Why not just post-hoc foot locking or PhySINK-style inverse kinematics? Locking, as in PHUMA, fixes the foot but not the camera. Under a drifting camera, the error moves into the body as leaning. We measure both foot sliding and camera error to show whether the error is removed or merely relocated.

Proposed Experiments

Controlled degradation. On EMDB, which provides ARKit camera ground truth, and on RICH, we progressively strip background texture from real pixels. The two operations are low-pass filtering and contrast compression of non-human regions. These change appearance but not geometry, so the camera ground truth remains valid. Metrics are:

  • camera absolute and relative trajectory error;
  • W-MPJPE and root translation error;
  • foot-skating rate (horizontal velocity of the contact point during stance);
  • acceleration error.

Real featureless capture. Because synthetic flattening may not resemble real blank walls, we also record walking, turning, and dance in rooms with blank walls. Following the XR Reality Check protocol, we use a handheld phone with an IMU and Vicon ground truth. We also repeat the ms-AIST close-up shots in which the subject covers most of the frame. HumanMM names these as a failure case and reports PA-MPJPE of 36.82 mm and RTE of 2.56 m.

Decisive checks. - The degradation curves should cross. Baseline camera error should rise sharply as texture falls, while ours rises slowly. - The metric-depth ablation should beat un-masked stance-foot patches, showing the depth term is essential. - Our method should beat the gyroscope-plus-learned-root baseline on the real captures. - If the anchor error stops shrinking with stance length, depth errors are temporally correlated. In that case we would add a learned temporal correction for the body network's depth.