A single camera cannot tell a toy car a meter away from a real car ten meters away. Both produce the same image, so every monocular 3D reconstruction is correct only up to an unknown multiplier. The problem is worse in dynamic scenes. Each independently moving object has its own unknown multiplier, separate from the background's, because its apparent motion could come from a small nearby object moving slowly or a large distant one moving fast. Current fixes borrow scale from something assumed known: the camera's height above a flat road, a typical human body, a typical object size, or a learned depth network's sense of what "looks" metric. This proposal uses a quantity that is known exactly and does not depend on appearance at all. Anything in free flight accelerates downward at 9.81 m/s², and video timestamps are metric in time. A tossed ball, a jumping person, or falling debris therefore carries a built-in ruler. The idea is to fit a constant-acceleration model to the object's trajectory, which is known only up to scale, and read off the multiplier that makes that acceleration equal g.
The cue matters most in exactly the cases where existing anchors break: objects that are airborne, not touching the ground, not human, or atypical in size. When the object is later caught or lands, the scale it carries can be passed to the thrower, the ground, and the camera trajectory. The result is a metric camera path and metric object geometry from plain RGB video, with no depth sensor, no inertial sensor, and no size prior.
Core Claim
During any interval of unsupported motion, the metric scale of a moving object is identifiable from its image trajectory alone, given camera intrinsics and frame timing. If the camera also accelerates, the scale of the static scene becomes identifiable too. A factor graph (a probabilistic optimization that combines many soft constraints over shared variables) can hold one scale variable per moving object and one for the background. Adding gravity factors on detected free-flight segments, and contact factors at catch and landing events, will recover dynamic-object scale more accurately than the anchors used today. This holds most strongly on objects that are unusually sized, not touching anything, or not human.
The mechanism fits in one equation. Let a scale-free visual odometry system give camera centers $\hat c(t)$ and rotations $R(t)$, so the true camera center is $s\,\hat c(t)$ for an unknown scene scale $s$. Let the object's own scale-free reconstruction give its centroid in camera coordinates as $s_o\,\hat u(t)$. For a rigid object, $\hat u(t)$ follows from its image position and its apparent size relative to a reference frame. The object's world position is $x(t) = s\,\hat c(t) + R(t)\,s_o\,\hat u(t)$. During free flight,
$$ s\,\ddot{\hat c}(t) + s_o\,\frac{d^2}{dt^2}\big[R(t)\hat u(t)\big] = \mathbf{g}, \qquad \|\mathbf{g}\| = 9.81\ \text{m/s}^2, $$
where $\mathbf{g}$ is the gravity vector. Its direction is estimated jointly or taken from the scene's vertical. The equation is linear in $(s, s_o)$, so each flight segment adds an overdetermined least-squares block.
Two practical consequences follow. If the camera barely accelerates, only $s_o$ is constrained, and scene scale must come from a contact event such as the landing. The recovered length scale also varies with $g\,\Delta t^2$, where $\Delta t$ is the frame interval, so a 10% error in the stated frame rate becomes roughly a 21% scale error. This makes frame timing a first-class nuisance variable.
Relation to Prior Work
Four lines of work motivate the proposal and bound it. The first gets scale from ground geometry. GroCo blends predicted depth with a plane derived from the known camera mounting height. Embodiment computes trigonometric "physics depth" on road pixels and extends it upward onto objects. DepthP+P builds camera height into a planar-parallax formulation. All three assume a flat, visible ground and a static scene. We use no camera-height assumption.
The second line passes scene scale to moving objects through contact. Ranftl et al. reconstruct each rigid moving body up to its own scale and lock it to the background by forcing its lowest boundary to touch the ground. Crowd4D does the same for human feet on a reconstructed non-planar terrain. These papers supply our contact factors. They also define the gap we target: both fail by construction for objects in the air. Our design reverses the direction of transfer, so the airborne object gives scale to the scene at landing.
The third line uses semantic size priors: human height distributions in Fei et al. and in SynCHMR, hand size in Zimmermann et al., a known target size in the MBZIRC balloon-popping system, and object dimensions queried from a large language model in 6D Object Pose Tracking in Internet Videos. These priors are biased whenever an instance departs from its class average. SynCHMR's own limits list children and atypical bodies as failure cases. Gravity has no such variance.
The fourth line supplies the machinery we reuse. PRISM-SLAM gets metric monocular SLAM from foundation-model depth through Plücker ray-distance factors. It filters scale in the log domain and softly down-weights dynamic regions. We adopt its factor-graph and log-scale structure. The difference is that we give dynamic objects their own scale variables instead of treating them only as disturbances. Metric3D v2 provides the appearance-based metric prior that gravity must be shown to complement. Its canonical-camera transform depends on known focal length, and that same dependency also limits our method.
We do not claim to solve dynamic 4D reconstruction. Systems like 4D-GS SLAM and USplat4D serve only as downstream consumers that we render at the recovered metric scale.
Research Program
The system has four parts. The first is a free-flight detector. It tracks each moving object, which is segmented by motion inconsistency with the static background in the manner of Ranftl et al.'s multibody decomposition. It then runs a sliding-window test comparing a constant-acceleration model against a general smooth-motion model on the scale-free trajectory $R\hat u$. Candidate segments must also be free of contact with other tracked bodies.
The second part is the gravity factor above. It carries a robust loss and an explicit drag-residual term, so that light, high-drag objects such as balloons and leaves are down-weighted rather than trusted.
The third part is a set of contact factors at catch, release, bounce, and landing. These tie $s_o$ to the scale of whatever body the object touches, whether the scene or a person.
The fourth part fuses these factors with foundation-model depth factors and anthropometric factors when humans are present. Everything is expressed in log-scale, so every cue is a soft vote with its own uncertainty.
Frame timing is estimated as a variable. It starts from container metadata and is regularized by any other periodic evidence available. An analysis track characterizes identifiability: the minimum flight duration and image-space extent needed for a given scale accuracy, and how often casual video actually contains usable flight segments. These are questions about the size of the opportunity, not just the method.
Baselines
The naive reference point is per-frame metric depth from Metric3D v2 with focal length from metadata, reading each object's scale off its median predicted depth. This is what a practitioner would do today. The file reports AbsRel of 0.044 on KITTI and 0.047 on NYUv2 for its ViT-Large model (Metric3D v2). No reported result was found on Lacuna for per-object scale accuracy on moving objects.
The strongest static-scene competitor is a PRISM-SLAM-style factor graph with log-domain scale filtering, run on identical video with identical depth-network weights. It reports an SE(3) ATE of 3.04 cm on TUM fr1/xyz without alignment (PRISM-SLAM). Our system reuses that graph, so any gain comes only from the added factors.
For dynamic-object scale, the strongest competitors are Ranftl et al.'s ground-contact prior and SynCHMR's anthropometric scaling for human subjects. Both run on the same tracks and segmentations.
A beat is defined as follows. We must reduce median per-object log-scale error, $|\log(\hat s_o/s_o^{\ast})|$, against the best of these baselines on unsupported and atypically sized objects, without worsening camera ATE on sequences that have no flight segments.
Three ablations isolate the key ingredient. The first removes the gravity factor. The second replaces 9.81 with a deliberately wrong value, such as 1.2g. The recovered scale should shift by the predicted factor of about 1.2, which shows the cue is actually driving the estimate rather than riding on the depth prior. The third swaps the detector for oracle flight segments to separate detection error from estimation error. All methods share tracks, depth weights, intrinsics, and a matched tuning budget on a held-out split.
Why not just…?
Why not just read each object's scale from a metric depth network? This is the real threat. On ordinary objects at ordinary distances, Metric3D v2 may be as good as gravity. Its accuracy rests on appearance matching typical scenes and on correct focal length, and PRISM-SLAM reports that naive use of such deterministic per-frame predictions is temporally unstable. Gravity is independent of both. The proposal therefore stands or falls on subsets where appearance misleads: scale models and toys, unusual objects, and missing or wrong EXIF data. The combined system must also match the network where the network is right. If gravity adds nothing on these subsets, the fusion factor is not worth keeping, and we will report that.
Why not just use known object sizes? Size priors from class averages or language models (6D Object Pose Tracking in Internet Videos) carry intra-class variance directly into the scale. The same objection applies to anthropometric priors on children in SynCHMR. Our evaluation includes a children-jumping subset where the height prior is biased and gravity is not.
Why not just use ground contact? Contact priors are correct when contact exists, and we keep them as factors. Ranftl et al. and Crowd4D cannot constrain an object while it is airborne, and they need a correct ground model first. Crowd4D's own limits note that a wrong initial mesh sinks the whole alignment. A contact-only ablation measures how much of the gain occurs during unsupported intervals.
Why not just use the phone's inertial sensor? When an accelerometer stream exists, visual-inertial scale as in Wofk et al. is better, and we would use it. The target here is archival and internet video, where no such stream exists. Gravity observed in the scene then plays the role of the missing accelerometer.
Proposed Experiments
No benchmark in the file contains labeled free flight, so evaluation combines one synthetic source and two real ones.
The synthetic set extends the controlled-motion design of the Instructive Synthetic Dataset and the Objaverse assets used by USplat4D. It renders physically simulated throws, drops, and bounces under swept camera motion, flight duration, object size, focal error, frame-rate error, and drag. This yields identifiability curves: scale error as a function of flight time and image extent.
The first real set is new captures of thrown dense objects and jumping people, recorded on a phone and scanned with a laser measure or depth sensor for ground truth. Depth is never given to the method.
The second real set is internet sports video containing regulation balls. The known ball diameter serves only as held-out ground truth for scale, never as an input.
We also run on BONN RGB-D Dynamic and TUM with depth withheld. This checks that the added factors do nothing harmful when no flight is present. For context only, 4D-GS SLAM reports 3.6 cm average ATE on BONN, but that system uses RGB-D input (4D-GS SLAM).
Metrics are per-object log-scale error, AbsRel of object depth, metric ATE of the camera without Sim(3) alignment, and the fraction of real clips containing a usable flight segment.
Three checks decide the question. First, the wrong-g ablation must shift scale by the predicted ratio. Second, gravity must beat Metric3D v2 per-object scale on the toys and miniatures subset, and SynCHMR on the children subset. Third, landing events must reduce camera ATE relative to the PRISM-style graph on sequences where both the camera and the object accelerate. If gravity fails the second check while passing the first, the cue is real but redundant, and the honest conclusion is that appearance priors already capture it.
References
- GroCo: Ground Constraint for Metric Self-supervised Monocular Depth
- Embodiment: Self-Supervised Depth Estimation Based on Camera Models
- DepthP+P: Metric Accurate Monocular Depth Estimation using Planar and Parallax
- Dense Monocular Depth Estimation in Complex Dynamic Scenes
- Crowd4D: Scene-Aware Monocular 4D Crowd Reconstruction
- Single View Physical Distance Estimation using Human Pose
- Synergistic Global-Space Camera and Human Reconstruction from Videos
- Learning to Estimate 3D Hand Pose from Single RGB Images
- Visually Guided Balloon Popping with an Autonomous MAV at MBZIRC 2020
- 6D Object Pose Tracking in Internet Videos for Robotic Manipulation
- PRISM-SLAM: Probabilistic Ray-Grounded Inference for Scale-aware Metric SLAM
- Metric3D v2: A Versatile Monocular Geometric Foundation Model for Zero-shot Metric Depth and Surface Normal Estimation
- 4D Gaussian Splatting SLAM
- Uncertainty Matters in Dynamic Gaussian Splatting for Monocular 4D Reconstruction
- Monocular Visual-Inertial Depth Estimation
- Monocular Dynamic Gaussian Splatting: Fast, Brittle, and Scene Complexity Rules