Methods that rebuild a 3D scene from video mostly assume the world holds still. When something moves, they either smear it into semi-transparent ghosts or, in 3D Gaussian Splatting (a scene made of millions of small colored 3D blobs that are rendered by projecting them onto the image), spawn stray blobs that hang in mid-air. Dynamic variants fix this by learning a motion field. They also need a reliable signal saying which pixels move because the object moves and which move only because the camera does. The strongest current signal comes from MotionGS. It predicts how each pixel would move under camera motion alone by reprojecting the depth the model itself renders, then subtracts that from the observed optical flow (the per-pixel 2D motion between frames). That signal is circular, though. Early in training, or with few views, the rendered depth is wrong, so the "camera-only" motion is wrong, so the supervision is wrong.
We propose to break that loop with a cue that needs no global depth at all: optical expansion, meaning how much a small image patch grows or shrinks between frames. Expansion measures the ratio of a point's new depth to its old depth. A static point seen from a known camera motion has two independent ways to reveal its depth. One is how far it slides in the image (parallax). The other is how much its patch expands. For a static point these must agree, and for a moving point they generally do not. The disagreement holds even in the classic blind spot where an object slides exactly along the line a static point would follow. This gives a per-pixel, depth-free test for "is this pixel static?" that is accurate from the first iteration. We use it to decide when the model's own depth can be trusted, to supervise motion, and to control where new Gaussians may be created.
Core Claim
Combining two local measurements with the camera motion yields a static-consistency residual that is independent of any reconstructed or predicted depth map. The two measurements are 2D flow and optical expansion; the camera motion is rotation $R$ and translation $t$. After removing camera rotation from the flow, triangulating the flow gives a parallax depth $Z_{\text{tri}}$. For a static point, the expansion network's motion-in-depth estimate $\tau = Z'/Z$ must then match the value implied by that depth and the forward component $t_z$ of the camera translation:
$$ r = \left|\log \tau_{\text{meas}} - \log\left(1 - \frac{t_z}{Z_{\text{tri}}}\right)\right| \Big/ \sigma, $$
where $\sigma$ is a per-pixel uncertainty that combines the expansion network's confidence with the conditioning of the triangulation. The triangulation is poorly conditioned near the epipole and at small baselines.
When the camera moves sideways ($t_z \approx 0$), the test reduces to "a static patch should not change scale." When the camera moves forward, both terms carry information.
The prediction is specific. Deformable 3DGS supervised by motion targets gated on $r$ should beat MotionGS, and the margin should be largest exactly where MotionGS is weakest: early iterations, sparse multi-view rigs, and objects moving along epipolar lines.
Relation to Prior Work
Our proposal sits on two lines of work.
The first supplies the method. Upgrading Optical Flow to 3D Scene Flow Through Optical Expansion shows that a local affine fit on small patches recovers motion-in-depth well. It reports a time-to-collision error of 4.21% against 6.94% for stereo on KITTI-15, at about 15 ms of overhead. That paper uses expansion to upgrade flow to scene flow. It never uses expansion as a consistency check against camera parallax, and it never feeds it into a reconstruction model.
The second line motivates the gap. MotionGS decouples flow using rendered depth and documents cascading failure in sparse-view settings. Three other systems lift or consume flow without disambiguating axial motion. De4D-SLAM lifts raw 2D flow to warm-start dynamic Gaussians. DynaSplat cross-references deformation variance with 2D flow. UA-4DGS seeds dynamic regions from off-the-shelf scene flow and depth. Our residual is complementary to all three, and it could be dropped into any of them.
NSFF also supervises 3D scene flow, with monocular depth hints. It inherits the scale inconsistency of those hints, and it trains for days per clip.
RobustSplat constrains the design. It shows that densification triggered by transient error causes floaters, and that turning densification off blurs the scene. So any gate we add must be selective, not global.
Research Program
The pipeline starts with a frame-pair stage. It computes flow with GMFlow, the flow estimator MotionGS uses, so the comparison stays fair. It computes expansion with the optical expansion network. It takes camera poses from the same COLMAP initialization and joint pose refinement that MotionGS uses. From these it produces the residual $r$ and a soft static probability $p_s = \exp(-r^2/2)$.
The reconstruction is the deformable 3DGS backbone that MotionGS builds on. The residual enters it in three places.
First, $p_s$ acts as a trust gate on the model's rendered depth. MotionGS-style camera-flow subtraction is applied only where the rendered depth agrees with $Z_{\text{tri}}$ on high-$p_s$ pixels. Elsewhere, static pixels are supervised to have zero object motion directly.
Second, on low-$p_s$ pixels, the normalized scene flow from expansion supervises the direction of the projected Gaussian motion, including its depth component. MotionGS's purely 2D Gaussian flow cannot express that component.
Third, densification driven by photometric error is allowed only where the error is not explained by motion. Concretely, we densify static Gaussians only in high-$p_s$ regions and dynamic Gaussians only in low-$p_s$ regions. This adapts RobustSplat's insight into a per-region rule rather than a global delay.
We will also analyze the test itself. We will measure how reliable $r$ is as a function of baseline, $t_z$, motion blur, and object rotation. Object rotation violates the expansion model, but such objects are dynamic, so a violation should still be flagged as "not static." The open question is whether $\sigma$ absorbs these effects, or whether the test produces false "dynamic" labels on static, textureless, or specular surfaces. NeRF-DS is the natural stress test for the specular case, given NeRF-DS.
Baselines
The naive reference is the Deformable-3DGS backbone with no motion guidance. MotionGS reports it at 23.61 dB PSNR on NeRF-DS. Alongside it we run raw-flow supervision, which MotionGS reports at 23.37 dB. These two numbers bound "do nothing" and "do the obvious thing."
The strongest direct competitor is MotionGS. It is built on the same backbone, it uses the same flow estimator, and it addresses exactly the camera-versus-object decoupling we target. It reports 24.54 dB mean PSNR on NeRF-DS and 24.8 dB on HyperNeRF. We will rerun it from its released configuration on identical frames, poses, flow inputs, iteration counts, and GPU. That way any gap reflects the motion signal, not the tuning. Beating it means a mean PSNR gain on NeRF-DS and HyperNeRF that exceeds its across-seed variation. It also means a clearly larger gain in the early-iteration and sparse-view regimes.
For casual monocular video we compare against UA-4DGS, which reports 15.25 masked PSNR on DyCheck against 14.14 for 4DGS. For multi-view sparse rigs, where MotionGS reportedly breaks, we compare against Gaussian-Flow, which reports 32.0 PSNR on Plenoptic Video.
Two ablations isolate the key ingredient. In the first, the expansion term is removed, leaving only an epipolar (Sampson) residual from flow and pose. In the second, the depth-trust gate is removed, and the expansion-derived mask is used as a fixed segmentation instead.
Why not just…?
Why not just delay MotionGS's flow loss until rendered depth has converged? This is the most serious threat. If a warm-up schedule recovers most of our gain on dense monocular benchmarks, the residual adds little there. We will run MotionGS with tuned warm-ups, giving it the same search budget. We expect warm-up to help on NeRF-DS and HyperNeRF, but not in sparse-view rigs. In those rigs, by MotionGS's own account, rendered depth may never become trustworthy. If warm-up matches us in sparse-view settings too, we will report that and drop the trust-gate claim.
Why not just use an epipolar residual to find moving pixels? It is cheaper and needs no expansion network. However, it is provably blind to objects whose 2D motion stays on the epipolar line, which is the projective confusion described in the research file. The first ablation tests this directly. We will report results separately on a subset of pixels where object motion is nearly epipolar-aligned, identified on the D-NeRF synthetic scenes using ground-truth motion. If the epipolar-only variant matches ours even there, expansion is not earning its cost.
Why not just back-project flow with an off-the-shelf monocular depth network? MotionGS already found that MiDaS-based camera flow underperforms native depth, because of scale drift and inconsistency between frames. Our residual uses neither kind of depth. As a check, we include a variant that gates on agreement with MiDaS depth instead of $Z_{\text{tri}}$.
Why not just mask movers with a segmentation model? 3DGS-HPC documents that such masks miss non-semantic motion, such as shadows and shifted chairs. Split4D reports that tracker-dependent lifting reaches only 0.593 mIoU on Multi-Human, against 0.893 without trackers. Masks also say nothing about motion in depth.
Proposed Experiments
The first decisive check is a curve of PSNR, and of camera-flow error against D-NeRF ground truth, versus training iteration, for MotionGS and for our method. The claim predicts that we lead most in the first few thousand iterations.
The second check uses sparse multi-view Plenoptic Video and N3DV. MotionGS should degrade there and we should not. On N3DV no per-scene number was found on Lacuna for DynaSplat, so we will rerun it.
The third check is the epipolar-aligned subset ablation described above.
As a secondary test of the densification gate, we will run on NeRF On-the-go and RobustNeRF. The question is whether $p_s$ resolves 3DGS-HPC's "rare object problem," where a static object seen only briefly is mistaken for a transient one. It should, because such an object has a near-zero residual.
On RobustNeRF, 3DGS-HPC reports a gain of 1.39–5.54 dB over 3DGS. No numeric result was found on Lacuna for RobustSplat on either benchmark, so we will rerun RobustSplat under our protocol.
Metrics are PSNR, SSIM, and LPIPS, plus masked PSNR on DyCheck. We also report the extra runtime of the frame-pair stage, so that any gain can be weighed against its cost.
References
- Upgrading Optical Flow to 3D Scene Flow Through Optical Expansion
- MotionGS: Exploring Explicit Motion Guidance for Deformable 3D Gaussian Splatting
- De4D-SLAM: Gradient-Isolated Static-Dynamic Decoupling for Monocular SLAM in Dynamic Environments
- DynaSplat: Dynamic-Static Gaussian Splatting with Hierarchical Motion Decomposition for Scene Reconstruction
- 4D Gaussian Splatting in the Wild with Uncertainty-Aware Regularization
- Neural Scene Flow Fields for Space-Time View Synthesis of Dynamic Scenes
- RobustSplat: Decoupling Densification and Dynamics for Transient-Free 3DGS
- NeRF-DS: Neural Radiance Fields for Dynamic Specular Objects
- Gaussian-Flow: 4D Reconstruction with Dynamic 3D Gaussian Particle
- 3DGS-HPC: Distractor-free 3D Gaussian Splatting with Hybrid Patch-wise Classification
- Split4D: Decomposed 4D Scene Reconstruction Without Video Segmentation