Feed-forward 3D reconstruction models take a few video frames and output a whole 3D scene in one network pass, with no per-scene optimization. They usually represent the scene with 3D Gaussians, which are small semi-transparent ellipsoids that can be rendered very quickly. When the input is a single moving camera watching a scene that also moves, these models render plausible images, but the geometry underneath is poor. The Gaussians are fat, cloudy blobs smeared along camera rays. The best-known remedy in optimization-based pipelines is anisotropic flattening, a penalty that squashes each ellipsoid's shortest axis toward zero so that it becomes a thin disk lying on the surface. The research file surfaces a real conflict here. A monocular SLAM system argues that fat Gaussians are useful because they act as an "error buffer" that absorbs wrong depth estimates (Pseudo Depth Meets Gaussian). Flattening removes exactly that buffer. In a dynamic monocular video, depth along the ray is often truly unknowable, because an object moving toward the camera looks the same as a nearer object standing still.
We propose to resolve the conflict rather than pick a side. Each Gaussian should be exactly as thick along its viewing ray as the model's depth error at that pixel is expected to be. It should be thin where parallax and multi-frame evidence pin depth down, and elongated where motion and monocular ambiguity leave depth open. We enforce this with one training objective that pairs a flattening penalty with a depth-likelihood term whose spread is the Gaussians' own rendered depth variance. Because the variance is computed from the rendered Gaussians themselves, thickness becomes a calibrated uncertainty signal produced by the geometry rather than by a separate prediction head. The result would be the first feed-forward dynamic model whose surfaces are thin where they can be trusted and honestly marked where they cannot.
Core Claim
Uniform flattening in a feed-forward dynamic model should hurt in exactly the regions where monocular depth is ambiguous: moving objects, low-parallax distant structure, and motion-occlusion boundaries. It should help everywhere else. A single loss that trades flattening against a depth likelihood lets the network find this trade-off per Gaussian. Concretely, let $D_p$ and $V_p$ be the alpha-composited mean and variance of depth along the ray through pixel $p$. $V_p$ grows both when individual kernels are thick along the ray and when several kernels are stacked at different depths. Let $D^*_p$ be a reference depth where one exists, $s_{\min,i}$ the shortest scale of Gaussian $i$, and $\alpha_i$ its opacity. We train with
$$ \mathcal{L} = \mathcal{L}_{\text{photo}} + \lambda_f \sum_i \alpha_i\, s_{\min,i} + \lambda_u \sum_p \left( \frac{|D_p - D^*_p|}{\sqrt{V_p}} + \tfrac{1}{2}\log V_p \right), $$
where the last term is a Laplace negative log-likelihood. The flattening term pushes every kernel thin. The likelihood term charges a heavy price when a thin kernel sits at the wrong depth, and it lets the kernel thicken instead. At the optimum, the along-ray spread tracks the expected absolute depth error. This follows because, for a Laplace likelihood, the scale that minimizes expected loss equals the mean absolute residual.
The claim can be checked in two ways. Uniform flattening should lose rendering quality specifically in ambiguous regions. The proposed loss should match uniform flattening's geometry in unambiguous regions while recovering the lost quality in ambiguous ones.
Relation to Prior Work
Three feed-forward systems already combine flattening with learned reconstruction, and all three assume a static world. Surf3R predicts flattened Gaussians from sparse, uncalibrated views and aligns their normals with rendered depth gradients. Flash-Mono predicts 2D surfels from streaming video for SLAM. Revisiting Depth Representations for Feed-Forward 3D Gaussian Splatting anchors Gaussian centers to a static pointmap prior but leaves their shapes unconstrained. We reuse the flattening and normal-alignment machinery from these methods. The difference is that we make the amount of flattening depend on how ambiguous the depth is, and we apply it to moving scenes.
On the dynamic side, BTimer predicts a full Gaussian scene frozen at any requested timestamp from monocular video, and its authors acknowledge that the resulting geometry contains floating blobs. L4GM does the same for isolated objects. 4RC factorizes video into base geometry plus a displacement field, without any surface primitives. BTimer is our backbone and main baseline. We keep its "bullet-time" formulation and change only the geometric objective.
DeSiRe-GS is the closest prior work in spirit. It already combines flattening with static/dynamic decomposition, but it optimizes each scene separately and relies on LiDAR and multi-camera rigs. It motivates our work by showing that flattening pays off in dynamic street scenes, and it serves as a per-scene upper reference. Finally, Pseudo Depth Meets Gaussian motivates the proposal from the opposite direction. Its error-buffer argument is the hypothesis we test and then turn into a mechanism, instead of treating it as a reason to give up on thin surfaces.
Research Program
We will build on a BTimer-style transformer that predicts pixel-aligned Gaussians at a queried timestamp. We add three things. The first is a rasterizer pass that composites the second moment of depth, which gives $V_p$ at negligible cost. The second is the flattening term together with Surf3R-style alignment between each flattened kernel's normal and the rendered depth gradient. The third is the likelihood term.
Depth references $D^*$ come only from data that has them. We use LiDAR on Waymo, sensor depth on TUM RGB-D and ScanNet, and ground truth on Replica. On clips without depth, the likelihood term is switched off and only the photometric and flattening terms act. The central empirical question is whether calibration learned on depth-bearing data transfers to clips without depth. Pretraining follows BTimer's recipe, starting on static RealEstate10K and DL3DV and then moving to dynamic video, at 256×256 input resolution.
The analysis is organized around a single stratification. For each test pixel we compute a parallax score from the context-camera baseline and depth. We also compute a motion label from ground-truth scene flow where available, or otherwise from the residual of a static render, as in DeSiRe-GS. Every metric is then reported separately for static high-parallax, static low-parallax, and dynamic pixels. Without this split, the error-buffer hypothesis cannot be seen in aggregate averages.
Baselines
The naive reference point is BTimer exactly as published: unconstrained 3D Gaussians and photometric loss only. On the NVIDIA Dynamic Scene dataset its authors report 25.82 dB PSNR with 150 ms reconstruction from 12 context frames (BTimer). We will retrain it with our data mixture and training schedule so that differences come only from the objective, and we will also report the published figure for reference.
The strongest direct competitor, and the most important one, is uniform flattening. This is the same backbone plus a Surf3R-style shortest-axis penalty and depth-gradient normal alignment, with the same depth supervision applied as a plain L1 loss and no variance term. It represents the obvious combination of existing parts. If our method does not beat it, the proposal has no content. Beating it means two things together. On dynamic and low-parallax pixels, we must show higher held-out PSNR and lower depth error. On static high-parallax pixels, we must show surface F1 and normal consistency that are no worse, within seed variance.
A second competitor is a decoupled uncertainty head. The network outputs a separate per-pixel uncertainty map and is trained with the same Laplace likelihood, while all kernels are flattened uniformly. This baseline isolates our key ingredient, which is that the uncertainty lives in the kernels' geometry. A third competitor is binary mask gating: flattening is applied at full strength outside a motion mask and switched off inside it. This tests whether continuous, parallax-aware thickness matters beyond the simple rule of not flattening movers.
All variants share the backbone, data, step count, and a five-value sweep over each loss weight on a held-out validation split, and each is trained with three seeds.
Per-scene references show how much quality the feed-forward speed costs. DeSiRe-GS reports 33.61 dB PSNR on Waymo, against 32.46 dB for PVG (DeSiRe-GS), but it uses LiDAR and per-scene training, so it is an upper reference rather than a competitor. For the static regime, Surf3R reports a surface F1 of 78.71 on ScanNet++ (Surf3R). We will evaluate our model under Surf3R's sparse-view protocol to confirm that the likelihood term does not erode static surfaces.
For 4RC, the file gives no reported result on Lacuna for depth or surface accuracy on our splits. The only number available is a relative 36% gain in the Average Percentage of Points tracking metric. We will therefore compare against its rendered depth only if its outputs can be obtained.
Why not just…?
Why not just add a flattening loss to BTimer? This is the real threat, which is why it is the main baseline. The Gap 2 tension predicts that uniform flattening is harmful only in ambiguous regions, so its failure should be invisible in scene-averaged PSNR and visible in the stratified split. If uniform flattening shows no drop in dynamic or low-parallax strata, the error-buffer hypothesis is false for feed-forward models. We would report that outcome, and the contribution would shrink to the finding that flattening transfers to dynamic feed-forward reconstruction for free.
Why not just mask moving objects and flatten everything else? Motion is only one source of ambiguity. Distant static structure with little parallax is ambiguous as well, and masks leak at boundaries, which is how 2D errors become 3D scars in modular pipelines (RobustNeRF, Distractor-free Generalizable 3D Gaussian Splatting). The mask-gated baseline settles this question. If it matches our method on every stratum, ambiguity is effectively the same as motion, and the simpler rule wins.
Why not keep fat Gaussians and predict uncertainty with a separate head? A separate head can be calibrated while the rendered geometry is still wrong. Nothing forces the kernels themselves to reflect doubt, so floaters survive. The decoupled-head ablation compares floater mass and surface F1 at equal calibration quality. We expect it to match our calibration but to lose on geometry in ambiguous regions, where its uniformly flat kernels commit to the wrong depths.
Why not refine BTimer's output with a few minutes of per-scene flattening optimization? Refinement gives up the 150 ms regime that motivates feed-forward work. It also still has to decide how much to flatten each kernel without an uncertainty signal. We will report a refinement variant with a matched time budget for context, not as the target.
Proposed Experiments
We evaluate on four benchmarks, each chosen for a specific role. NVIDIA Dynamic Scene measures held-out view synthesis, reported as PSNR, SSIM, and LPIPS per stratum. Waymo is the geometry testbed for dynamic scenes, using LiDAR depth for absolute relative depth error and Chamfer distance on static and dynamic points separately. TUM-dynamics provides indoor depth-bearing dynamic sequences for calibration. ScanNet++ and Replica serve as static controls, measured by F1 score.
We add two geometry-specific metrics. The first is floater mass, the fraction of opacity lying more than a fixed distance from the reference surface. The second is calibration, measured as the Spearman correlation between $\sqrt{V_p}$ and the actual absolute depth error. We also report coverage of the ±1 and ±2 spread intervals, evaluated on Waymo and TUM data held out from training depth supervision, to test whether calibration transfers.
Three results would be decisive. The first is a measurable PSNR drop for uniform flattening, relative to unconstrained BTimer, confined to dynamic and low-parallax strata. The second is that our method removes that drop while matching uniform flattening's static-stratum F1. The third is a positive, well-calibrated correlation between thickness and error on sequences with no depth supervision. If the first result fails, the premise fails. If the second or third fails, the mechanism fails. Either way the outcome is informative.
References
- Pseudo Depth Meets Gaussian: A Feed-forward RGB SLAM Baseline
- Surf3R: Rapid Surface Reconstruction from Sparse RGB Views in Seconds
- Flash-Mono: Feed-Forward Accelerated Gaussian Splatting Monocular SLAM
- Revisiting Depth Representations for Feed-Forward 3D Gaussian Splatting
- Feed-Forward Bullet-Time Reconstruction of Dynamic Scenes from Monocular Videos
- L4GM: Large 4D Gaussian Reconstruction Model
- 4RC: 4D Reconstruction via Conditional Querying Anytime and Anywhere
- DeSiRe-GS: 4D Street Gaussians for Static-Dynamic Decomposition and Surface Reconstruction for Urban Driving Scenes
- RobustNeRF: Ignoring Distractors with Robust Losses
- Distractor-free Generalizable 3D Gaussian Splatting