A single moving camera can work out how far away a static object is by watching it from two places. That is triangulation, and it fails for things that move. When a person walks across the view of a drone with one camera, every image pins down only the direction to the person, not the distance. A near person walking slowly and a far person walking fast trace exactly the same sequence of directions. If the camera itself moves in a straight line at constant speed, no amount of watching separates these hypotheses, because sideways camera motion is canceled by an unknown object velocity. The ambiguity breaks only when the camera changes its motion relative to the target. A turn, a speed-up, or a vertical bob gives the object hypotheses different predicted directions. In other words, the camera's own path decides whether a moving object's distance can be measured at all.
We propose an active planner for a monocular robot in a scene with moving agents. It chooses short camera maneuvers specifically to make the depth and velocity of moving objects observable. It ties those object estimates to a static map whose global scale is fixed separately, by combining depth predictions from a pretrained network with multi-view geometry. The planner picks motion segments by how much they shrink the uncertainty of a small per-object motion state, using a closed-form information measure that is cheap enough to run online. The result is that the robot moves so as to measure the moving objects, rather than ignoring them as noise or chasing them as unexplained error.
Core Claim
The central claim is threefold. First, depth ambiguity for moving objects seen by one camera is a planning problem and not only an estimation problem. Second, it reduces cleanly to two separable pieces. One is the global scale of the static map, which recent geometry-plus-prior methods already handle passively. The other is the range of each moving object relative to that map, which is unobservable under unaccelerated relative motion and becomes observable through camera maneuvers. Third, a planner that maximizes information about object motion state will reach a given metric error on moving objects in markedly fewer frames than two simpler policies: static-scene active planners, and parallax-maximizing heuristics. It will do so without degrading the static map or camera tracking.
The degeneracy can be stated precisely. Let $\mathbf{o}_0, \mathbf{v}$ be a moving object's initial position and constant velocity, and $\mathbf{c}_t$ the camera center at time $t$. The camera observes only the bearing of the relative position $\mathbf{q}_t = \mathbf{o}_0 + t\mathbf{v} - \mathbf{c}_t$. Suppose the camera also moves at constant velocity, $\mathbf{c}_t = \mathbf{c}_0 + t\mathbf{w}$. Then the relative state $(\mathbf{o}_0 - \mathbf{c}_0,\ \mathbf{v} - \mathbf{w})$ can be multiplied by any $k>0$ without changing a single bearing:
$$\frac{\mathbf{q}_t}{\|\mathbf{q}_t\|} = \frac{k\,\mathbf{q}_t}{\|k\,\mathbf{q}_t\|}, \qquad \mathbf{q}_t = (\mathbf{o}_0-\mathbf{c}_0) + t(\mathbf{v}-\mathbf{w}).$$
Because the camera path is known from tracking, a non-constant $\mathbf{c}_t$ cannot be rescaled along with the object, and the family collapses. The camera path is known up to the static map scale, which is why the two subproblems separate. This is a classical fact from bearings-only target tracking. It has not been used in learned active reconstruction, and it predicts something concrete: "move sideways for parallax", which is the right instinct for static scenes, is provably useless against a constant-velocity target.
Relation to Prior Work
Two papers come closest to the full triad, and each relaxes one leg.
Next Best View Selections for Semantic and Dynamic 3D Gaussian Splatting computes Fisher information over a deformation network to choose camera views and timestamps in dynamic scenes. It uses calibrated multi-camera rigs with known metric scale and scores appearance and deformation parameters, not object range. Our proposal targets exactly the quantity that is free in its setting: depth along the ray under monocular capture. We plan camera motions rather than selecting from a fixed pool of views.
PRISM-SLAM makes the monocular map scale identifiable by entering foundation-model depth into a factor graph as probabilistic ray-distance factors. It tracks scale with a log-domain Kalman filter and softly gates dynamic regions out of the optimization. It is passive, and it discards moving objects rather than estimating their depth. We reuse its scale machinery as our static layer and add what it explicitly leaves out: estimating moving agents and choosing a trajectory for that purpose.
The active-view literature supplies our method and also shows us the trap to avoid. FisherRF shows that Fisher information for view selection depends only on geometry and model state, not on unseen pixel colors, so it can be computed before moving. Its fast diagonal approximation is our template for the static-coverage term. Creat3r, ActiveNeRF, Active Neural Mapping at Scale and ActLoc all assume a static world. ActLoc names dynamic agents as future work. Creat3r's triangulated scaffold is corrupted by points that move. These systems motivate the proposal: they read photometric inconsistency as unscanned geometry. RobustSplat shows the reconstruction-side version of this failure, where transient residuals trigger spurious densification. Giving moving objects an explicit motion state is our answer to that confusion. Moving objects then explain their own residuals instead of inflating static uncertainty.
On the dynamic side, De4D-SLAM and WildGS-SLAM constrain our design. Both reach strong tracking on dynamic benchmarks by isolating or down-weighting movers. De4D-SLAM runs at 0.47 FPS and lifts optical flow to 3D velocity, which it reports degrades when the camera moves parallel to movers. That is the degeneracy above in practice. We therefore keep the planning state deliberately small and do not plan over a full 4D Gaussian model. VDNeRF reports that ego-motion and scene flow trade off in vision-only dynamic reconstruction, which is further evidence the ambiguity bites in real data. Crowd4D resolves crowd depth by snapping feet to a ground mesh. We include ground contact as an optional factor but do not rely on it, because Crowd4D notes it fails on unmapped or non-planar terrain. Finally, Metric3D v2 supplies the per-frame metric depth prior. Its dependence on known focal length and its single-frame nature are why we treat its output as a noisy measurement with learned variance rather than as truth.
Research Program
The estimator has two layers. The static layer is a PRISM-style factor graph: camera poses, sparse static landmarks, and a global log-scale fed by foundation-model depth factors. Dynamic pixels are gated out using temporal depth consistency. The dynamic layer keeps, for each tracked moving agent, a small state: position, velocity and optionally a physical height. It is updated by three kinds of measurement. The first is bearings to tracked points on the object, from 2D tracking or segmentation. The second is the foundation model's depth for those pixels, with a variance we calibrate per depth range, because depth on thin or same-speed objects is where these models fail, as LiDARTouch documents. The third is a ground-contact constraint when a static ground estimate exists.
The planner works over a receding horizon. It samples a library of feasible camera motion primitives (straight, arcs, accelerations, vertical offsets) and predicts the information each would yield over the next $H$ steps. For the dynamic layer this is a sum of bearing Jacobians evaluated along predicted relative paths, which is closed form and costs microseconds per object. The score is
$$\mathbf{u}^* = \arg\max_{\mathbf{u}} \Big[\log\det\big(\Sigma_{\text{obj}}^{-1} + \textstyle\sum_{\tau=1}^{H} J_\tau(\mathbf{u})\big) + \lambda\, G_{\text{static}}(\mathbf{u})\Big],$$
where $\mathbf{u}$ is a candidate camera motion, $\Sigma_{\text{obj}}$ is the current object-state covariance, $J_\tau$ is the predicted bearing-plus-depth-prior information at step $\tau$, and $G_{\text{static}}$ is a FisherRF-style coverage and scale gain on the static map, weighted by $\lambda$. The predictions also account for occlusion and keep-in-view constraints.
We will analyze when maneuvers matter. The analysis predicts that the benefit grows as the depth prior gets worse (far objects, unusual sizes, missing focal metadata) and vanishes when the prior is already tight. We will test that prediction directly.
Baselines
The naive reference is a passive monocular estimator on a fixed trajectory: straight-line or pre-recorded motion with the same two-layer estimator. It isolates what estimation alone achieves.
The strongest existing active method on the static side is FisherRF, run with dynamic pixels masked. This is the natural "mask and plan" pipeline. FisherRF reports 23.68 dB PSNR with 10 views on Blender, against 22.49 dB for random selection.
The strongest dynamic active method is Next Best View Selections for Semantic and Dynamic 3D Gaussian Splatting. We adapt it to monocular input by withholding rig calibration scale. It reports 33.60 dB PSNR on Replica against 29.66 dB for random. For the question that matters here, the metric position and velocity error of moving objects under active monocular capture, no reported result was found on Lacuna. All methods will therefore be re-run by us.
For the passive estimation backbone, PRISM-SLAM's reported unaligned trajectory error of 3.04 cm on TUM fr1/xyz is the target our static layer must match. We check this to confirm that adding the dynamic layer does not hurt tracking.
Two ablations isolate the key ingredient. The first is a parallax heuristic: always move perpendicular to the line of sight of the most uncertain object. The second is our planner with the dynamic information term removed, so $\lambda$-only.
Fairness rules apply to all comparisons. Every policy uses the same estimator, the same tracker and depth model, the same motion-primitive library, the same frame budget, and the same compute per planning step. Hyperparameters are tuned on held-out scenes with an equal search budget per method. Beating the baselines means two things at once. First, at least a 2× reduction in frames needed to bring moving-object range error under 10% of true range. Second, no statistically significant loss in static trajectory error or static rendering quality.
Why not just…?
Why not just mask movers and run a static active planner? This removes the confusion where the planner chases movers, but it never estimates mover depth. De4D-SLAM argues such blind spots are dangerous for navigation. Split4D shows that mask-tracker pipelines are brittle: the tracker-based SA4D reaches 0.592 mIoU against Split4D's 0.893 on Multi-Human. The masked-FisherRF baseline will show this directly, as no improvement in object error over passive capture.
Why not just trust per-frame metric depth from a foundation model? This is the real threat. If Metric3D v2 depth on moving people is accurate enough, maneuvers add little. The experiment that settles it stratifies scenes by depth-prior error, using distance bins, withheld focal length, and atypical object sizes. If the active gain is negligible in every stratum, we will report that the problem is solved by priors. If our analysis is right, the gain should concentrate where the prior is weak. Even in the best case, filtering the prior over time already sits inside our passive baseline.
Why not just maximize parallax? For static points this is close to optimal. For a target moving at constant velocity, constant sideways motion is exactly the unobservable case in the equation above. The parallax ablation should therefore help on slow or stationary agents and fail on steadily walking ones. Matching it on walkers would falsify our central claim.
Why not just add a cheap LiDAR? For many platforms this is the right engineering answer. LiDARTouch removes infinite-depth failures with 4 beams. It still misses thin structures and needs a PnP solve, which degrades in featureless scenes. Our target is platforms where a single camera is the only sensor, such as small aerial vehicles and wearables. We do not claim the method beats LiDAR.
Proposed Experiments
There is no existing benchmark with active control, a monocular camera, moving agents and metric ground truth, so we build three settings.
First, a controlled simulator. We insert scripted walkers following constant-velocity, piecewise and random-walk motion into Habitat scenes from Gibson and Matterport3D, where Active Neural Mapping at Scale and FisherRF report coverage. Metric ground truth is exact here.
Second, an offline active benchmark on Neu3D. The "camera" switches only between adjacent rig cameras at each timestamp, as in the dynamic NBV work, which produces a constrained virtual trajectory. Calibration scale is hidden from the estimator. Reference human positions come from triangulating across all rig cameras.
Third, a passive sanity check on Bonn Dynamic and TUM dynamic sequences. Here we only verify that the two-layer estimator tracks as well as its PRISM-style backbone, compared against De4D-SLAM's reported 1.95 cm on Bonn.
Metrics are as follows. For moving objects: range and velocity error in meters, frames until range error falls under 10%, and calibration of the predicted covariance (normalized estimation error). For the static map: trajectory error, the global scale error, and rendering PSNR. Also planning latency.
The decisive checks follow from the analysis. Our planner should beat the parallax heuristic specifically on constant-velocity walkers and tie it on near-stationary ones. The active gain should rise monotonically as depth-prior error increases. Removing the dynamic information term should reproduce the passive baseline's object error. If the first check fails, the thesis is wrong. If the second fails, the honest conclusion is that metric-depth priors have already absorbed the problem.
References
- Next Best View Selections for Semantic and Dynamic 3D Gaussian Splatting
- PRISM-SLAM: Probabilistic Ray-Grounded Inference for Scale-aware Metric SLAM
- FisherRF: Active View Selection and Mapping with Radiance Fields Using Fisher Information
- Creat3r: Confidence Reaggregation for Exploration-aware Active 3D Reconstruction
- ActiveNeRF: Learning Where to See with Uncertainty Estimation
- Active Neural Mapping at Scale
- ActLoc: Learning to Localize on the Move via Active Viewpoint Selection
- RobustSplat: Decoupling Densification and Dynamics for Transient-Free 3DGS
- De4D-SLAM: Gradient-Isolated Static-Dynamic Decoupling for Monocular SLAM in Dynamic Environments
- WildGS-SLAM: Monocular Gaussian Splatting SLAM in Dynamic Environments
- VDNeRF: Vision-only Dynamic Neural Radiance Field for Urban Scenes
- Crowd4D: Scene-Aware Monocular 4D Crowd Reconstruction
- Metric3D v2: A Versatile Monocular Geometric Foundation Model for Zero-shot Metric Depth and Surface Normal Estimation
- LiDARTouch: Monocular metric depth estimation with a few-beam LiDAR
- Split4D: Decomposed 4D Scene Reconstruction Without Video Segmentation