3D Gaussian Splatting (3DGS) builds a scene out of millions of small colored blobs ("Gaussians"). During training it grows new blobs wherever the rendering error pushes hard on existing ones. That growth rule quietly assumes the scene is static. When people, cars, or shadows move through the capture, every photo containing them produces a large error. The optimizer answers by spawning blobs that stay suspended in empty space long after the distractor has gone. RobustSplat documented this failure. Its fix is to postpone growth until the static scene settles, then bootstrap masks of the moving content. A separate line of work shows that the growth rule also fails on static but geometrically irregular scenes. There, errors over-feed flat regions and starve fine, sparse structure (CDC-GS, LeGS). Neither line handles both problems at once. The static-scene fixes will read moving edges as fine detail and grow blobs there. The distractor fixes work on fixed pixel patches and ignore how unevenly the blobs are distributed in 3D.
We propose a single change to the growth rule itself. Standard 3DGS decides to split or clone a blob by averaging its error gradient over the views that see it. We instead ask whether most of those views agree the blob needs more detail, using a robust statistic such as a median over views. The threshold is corrected for how many views see the blob and how crowded its 3D neighborhood is. The logic is simple. A static region that is under-reconstructed looks wrong from nearly every view that sees it, while a transient looks wrong only from the handful of frames in which it appears. The correction for crowding and visibility is where point cloud irregularity enters. Distant or rarely seen static structure is seen by few views and sits among sparse blobs, so a naive agreement rule would starve exactly the regions that irregularity already under-serves.
Core Claim
Densification can separate transient motion from genuine static complexity without any semantic mask, provided the densification signal is a consensus over views rather than an average, and provided the threshold accounts for how irregularly both views and primitives are distributed.
Concretely, let $g_{i,v}$ be the magnitude of the screen-space position gradient of Gaussian $i$ in training view $v$, and let $V_i$ be the set of views where $i$ is visible. Vanilla 3DGS densifies when the mean of $g_{i,v}$ over $V_i$ exceeds a global constant. We densify when
$$ Q_q\big(\{g_{i,v}\}_{v \in V_i}\big) > \tau\big(\rho_i, |V_i|\big), $$
where $Q_q$ is the $q$-th quantile across views (the median when $q=0.5$). Here $\rho_i$ is the local primitive density, measured as the inverse of the geometric-mean distance to the $K$ nearest Gaussians, following the density measure of CDC-GS. The threshold $\tau$ drops for sparse neighborhoods and for low visibility counts.
A transient that occupies fewer than a fraction $1-q$ of a blob's views cannot move the quantile. This gives a breakdown-point guarantee of the same kind that makes trimmed estimators robust. It is the densification-side counterpart of the trimmed photometric loss in RobustNeRF, which trims pixels in the loss but leaves the growth rule alone.
The new part is moving robustness from the loss into the growth decision and conditioning it on 3D irregularity. The closest existing work, RobustSplat, handles growth by timing (delay) rather than by per-primitive, cross-view evidence.
Research Program
The first component is instrumentation. We will log $g_{i,v}$ per view during 3DGS training on distractor-laden scenes and on the same scenes with the distractors removed. This tests the premise directly: we can check whether floater-producing blobs really have heavy-tailed, few-view gradient profiles while blobs in under-reconstructed static regions have broad support. Storing per-view gradients is memory-heavy, so the implementation will keep streaming quantile sketches per Gaussian over each densification window. This keeps the overhead linear in the number of Gaussians.
The second component is the threshold $\tau(\rho, |V|)$. We will compare two forms. The first is a closed-form correction: the standard error of a quantile grows roughly as $1/\sqrt{|V|}$ for fixed spread, which motivates relaxing the threshold for rarely seen blobs. The second is a small learned calibration fit on scenes with clean ground truth and then frozen.
The third component asks whether consensus gating makes mask bootstrapping unnecessary or merely complementary. We will plug the rule into a plain robust-loss 3DGS and into RobustSplat's pipeline with its delay removed.
Baselines
The naive reference is vanilla 3DGS with default mean-gradient densification, trained on distractor-laden captures. 3DGS-HPC reports that it improves PSNR by 1.39 to 5.54 dB over native 3DGS on the RobustNeRF dataset, which frames the size of the gap. A second naive point is 3DGS with densification disabled. RobustSplat reports that this removes floaters but blurs background detail, and we will reproduce that trade-off.
The strongest competitor for this specific question is RobustSplat. Its delayed growth and scale-cascaded masks target exactly the interaction between densification and transients. The file gives no scene-level numbers: no reported result found on Lacuna for RobustSplat PSNR on RobustNeRF or NeRF On-the-go. We will therefore rerun its public configuration ourselves.
3DGS-HPC is the strongest mask-based competitor. Its patch classifier calibrates DINOv2 feature thresholds using a mixture model of color residuals. We rerun it under its reported budget of roughly 13 to 17 minutes on an RTX 4090.
For NeRF-based reference points, RobustNeRF reports 30.87 dB versus 25.22 dB for Mip-NeRF 360 on its distractor scenes. NeRF On-the-go reports 21.41 dB versus RobustNeRF's 20.54 dB on Patio.
All Gaussian methods will use the same SfM initialization, iteration count, GPU, and a matched hyperparameter sweep of equal size. Final Gaussian counts will be reported alongside quality metrics.
Beating the strongest baseline means one of two outcomes on the RobustNeRF and On-the-go scenes. Either consensus gating, with the same loss as RobustSplat and no delay, matches or exceeds RobustSplat's PSNR, SSIM, and LPIPS with fewer Gaussians. Or it improves RobustSplat when added on top.
Three ablations isolate the ingredients: mean-gradient versus quantile gating, the quantile rule with a constant threshold versus the density- and visibility-aware $\tau$, and the value of $q$.
Why not just…?
Why not just mask the loss with transient masks, so masked pixels never send densification gradients? This is the most serious alternative, and many robust methods already do it implicitly. The weakness is timing. RobustSplat and 3DGS-HPC both report that masks are unreliable early in training, when semantic features are unstable and textureless static regions get carved away. Early training is precisely when most densification happens. We will run mask-gated densification using 3DGS-HPC's masks as a direct baseline. If it matches consensus gating, the honest conclusion is that masks suffice. We would then report consensus gating only as a mask-free fallback.
Why not just use CDC-GS or LeGS on top of a robust loss? Their complexity signals are computed from images (wavelet energy) or from per-primitive rendering sensitivity. A moving edge has high wavelet energy and high sensitivity in the frames where it appears. Without a cross-view consensus step, we expect them to grow blobs on distractors. The experiment below measures this directly.
Why not just delay densification, as RobustSplat does? Delay treats all blobs the same way. Distractors that persist through much of a capture, as in the On-the-go sequences, are still present when growth resumes. Meanwhile, delay costs static detail, a trade-off RobustSplat itself acknowledges. Consensus gating and delay can also be combined, and the combined configuration will be tested.
Why not just use the median without any density correction? This is our own method's simplest form, so it is an ablation rather than a rival. The prediction is that it will under-densify distant, rarely seen static structure. If it does not, the irregularity component is unnecessary, and we will say so.
Proposed Experiments
The main evaluation uses the RobustNeRF and NeRF On-the-go datasets, reporting PSNR, SSIM, and LPIPS against clean held-out views. These are the benchmarks used by RobustNeRF, NeRF On-the-go, RobustSplat, and 3DGS-HPC.
The decisive controlled check composites synthetic transients into the static Mip-NeRF 360 captures, where clean-trained reconstructions exist. We vary two things: the fraction of views each distractor occupies, and whether it falls on sparse or dense regions. The quantile rule predicts a sharp robustness threshold near a view fraction of $1-q$, and mean-based and image-complexity rules should show none. We also measure the number and total opacity of Gaussians lying inside the distractor's space-time footprint after training. This gives a direct floater count rather than inferring floaters from PSNR.
To confirm the rule does not harm static scenes, we train on clean Mip-NeRF 360. The target is to stay within noise of CDC-GS's reported 28.02 dB at a comparable Gaussian count, where it reports 2.53M Gaussians against 27.79 dB and 2.59M for vanilla 3DGS.
Finally, we stratify the error of every method by visibility count and by $K$NN density. This tests the irregularity claim: the density-aware threshold should recover quality specifically in the low-visibility, low-density bins where the uncorrected median is predicted to fail.
References
- RobustSplat: Decoupling Densification and Dynamics for Transient-Free 3DGS
- Reframing Gaussian Splatting Densification with Complexity-Density Consistency of Primitives
- Beyond Heuristics: Learnable Density Control for 3D Gaussian Splatting
- RobustNeRF: Ignoring Distractors with Robust Losses
- 3DGS-HPC: Distractor-free 3D Gaussian Splatting with Hybrid Patch-wise Classification