Voting in 3D to Keep Thin Wires When Filtering Moving Distractors

Casual 3D capture almost never happens in a still world. People walk through the shot and objects get moved. Methods that reconstruct a static scene from such footage cope by finding image regions that the model cannot explain and ignoring them. They make this decision in 2D, patch by patch, on the assumption that a moving thing is a solid blob. Thin static structures break that assumption. Wires, fence links, chair legs and antenna masts cover a sliver of each image. Early in training they are poorly fit, so they look exactly like unexplained error. They get masked out, and once masked they are never learned. We propose to move the static-versus-moving decision from image patches into 3D space. Each unexplained pixel casts a vote along its camera ray into a sparse 3D grid, and that grid can be read or written in constant time through a hash table, a lookup that finds a cell's storage directly from its integer coordinates. A thin wire's votes pile up in the same few cells from views spread across the whole capture. A moving person's votes either scatter or cluster only within a short time window. Lagrangian terms aside, the point is simple: the geometry of multi-view ray intersection separates "thin and static" from "transient" better than 2D shape does.

This matters because the failure is invisible in standard scores. The drone benchmark DDOS names it the "pixel count trap": depth models reach strong global error while failing almost completely on power lines. Distractor-robust reconstruction is headed toward the same trap. The strongest methods report gains in whole-image quality while conceding in their own limitations that they erase thin geometry.

Core Claim

The claim is that three signals, taken together, separate thin static structures from transient distractors far more reliably than any 2D patch statistic. The first is whether residual rays from many views converge on the same 3D location. The second is how spread out in capture time those views are. The third is whether the colors they carry agree. A hashed voxel store makes this 3D evidence cheap enough to maintain during training. Because the evidence lives in space rather than on individual primitives, it can also tell the reconstruction where to add geometry that does not exist yet. That second use is what fixes the "delayed growth" failure.

Concretely, take a 3D Gaussian Splatting model. This representation stores the scene as millions of small colored ellipsoids that are projected ("splatted") onto the image. At each training step, pixels whose photometric residual exceeds an adaptive threshold are traced along their rays. The traversal uses a standard voxel-walking algorithm with cell size $\Delta$. Each visited cell, keyed by $(\lfloor x/\Delta\rfloor, \lfloor y/\Delta\rfloor, \lfloor z/\Delta\rfloor)$ and hashed into a fixed-size GPU table, accumulates a few quantities: a residual-weighted vote count, the number of distinct camera views contributing, a running mean and variance of the residual colors, and the minimum and maximum capture timestamp of contributing views. A single ray crosses many cells. Only at a real, persistent surface do rays from different viewpoints meet, so vote density concentrates there. We score each cell as a thin-static candidate when three conditions hold together: votes come from at least $m$ distinct views, the timestamps of those views cover a large fraction of the capture, and the residual colors are consistent.

A candidate cell has two consequences. Pixels whose rays pass through it with high residual are exempted from the distractor mask. The cell is also allowed to seed new Gaussians, elongated along the principal direction of the ray intersections, even while global densification is still paused. Cells with high vote density but a narrow time span are treated as transient. We leave the scoring deliberately simple, with thresholds tied to dataset statistics, so that gains can be attributed to where the evidence lives rather than to a learned classifier.

Relation to Prior Work

The motivating evidence comes from distractor-robust reconstruction. RobustNeRF trims the highest-residual half of pixels and smooths the resulting masks over 16×16 patches. 3DGS-HPC classifies 16×16 patches as static or transient by thresholding self-supervised (DINOv2) features, with the threshold calibrated by a color-residual mixture model. Its authors state that extremely thin structures can be over-suppressed by this patch approach. NeRF On-the-go predicts per-pixel uncertainty from DINOv2 features. All three decide per image. We keep their robust loss as the outer loop but override its mask with 3D evidence.

RobustSplat is the closest method and also the one most in tension with us. It shows that Gaussian densification, the step that splits and clones ellipsoids where error is high, turns transient error into permanent floating geometry. Its remedy is to delay densification and to bootstrap masks from low resolution upward. It reports that disabling densification entirely gives overly smooth results. We accept its diagnosis. We argue, however, that its remedy withholds capacity exactly where wires need it, and our voxel seeding supplies that capacity selectively. Split4D models dynamics explicitly with per-Gaussian velocities. We do not aim to reconstruct the moving content, only to protect static thin geometry from being discarded with it.

The spatial-indexing work supplies the machinery. Voxel Map for Visual SLAM and Voxel Transformer show that hashing occupied cells decouples lookup cost from scene extent. Grid-GCN shows that grid neighbor queries replace costly tree-style queries with a reported 50× speedup in data structuring. Grid-GCN also documents the main risk we inherit: sensitivity to the cell size $\Delta$. PermutoSDF uses hashed lattices to push fine detail into an implicit surface. It concedes that thin disconnected strands remain unresolved and has no distractor handling. We use hashing for evidence bookkeeping rather than as the scene representation, which avoids the thin-sheet instability of signed distance fields described by Curriculum DeepSDF. SAGS regularizes Gaussians with a nearest-neighbor graph to preserve fine structure but assumes a static scene. Our method adds no graph and targets the dynamic regime.

Research Program

The build is a residual ray-voting module attached to an open 3D Gaussian Splatting codebase. It is implemented as a GPU kernel that traverses only above-threshold pixels. We expect these to be a few percent of pixels, so the cost should be a modest fraction of a training step. We will measure this rather than assume it.

The analysis has three parts. The first characterizes vote geometry on synthetic scenes where ground truth separates thin static, bulky static, and transient content. We will plot view count against time span for each class to test whether the two signals are actually separable. The second sweeps the cell size $\Delta$ and the table size to map the quantization and collision trade-off, including a two-level variant that refines cells only where votes concentrate. The third tests the known hard cases the file flags: static objects seen in only one or two views, which 3DGS-HPC calls the rare-object problem; specular surfaces; and clutter that stays put for a long stretch before moving. We expect the method to help the first only weakly, since it has few votes by construction. The time-span test may confuse long-stationary clutter with static geometry, and we will report that honestly.

Baselines

The naive reference is vanilla 3D Gaussian Splatting trained on all pixels. It shows how much damage distractors do at all. 3DGS-HPC reports gains of 1.39 to 5.54 dB PSNR over native 3D Gaussian Splatting, which bounds that gap.

The strongest patch-based competitor is 3DGS-HPC itself, because it is fast, mask-free, and explicitly admits the thin-structure failure we target. The strongest densification-aware competitor is RobustSplat, because its delayed growth and scale-cascaded masks are the closest existing attempt to protect fine static detail under distractors. For RobustSplat the file gives only qualitative claims of best results on RobustNeRF and On-the-go, so there is no reported result found on Lacuna for RobustSplat's exact scores. On the NeRF side we include RobustNeRF, which reports 30.87 dB PSNR against 25.22 dB for Mip-NeRF 360 on its benchmark RobustNeRF. We also include NeRF On-the-go, which reports 21.41 dB against RobustNeRF's 20.54 dB on the Patio scene NeRF On-the-go.

For fairness, we will rerun every Gaussian-based baseline from public code on the same images, poses, resolution, and iteration count, on a single RTX 4090 class GPU. Each gets an equal hyperparameter search budget over its mask threshold or trimming fraction, and a search over its own thresholds, and we report both its published setting and its tuned setting. Our method is implemented on top of 3DGS-HPC's pipeline, so the only difference is the 3D override.

The ablations isolate the key ingredient. We remove the time-span test, keeping only view convergence. We remove seeding, keeping only mask exemption. We replace hashed cells with per-Gaussian residual histories, which tests whether evidence must live in space. We replace the hash with an octree of equal resolution, which tests whether constant-time access matters for wall-clock cost.

Beating the baselines means two things at once. Global PSNR must stay within 0.2 dB of the best baseline on each scene, or above it. Thin-structure metrics, defined below, must improve by a margin larger than the seed-to-seed spread across three seeds.

Why not just…?

Why not just shrink the patches in 3DGS-HPC to 4×4 or classify per pixel? This is the most serious threat, and we treat it as a baseline rather than dismiss it. The 16×16 patches exist because single pixels are noisy, and per-pixel residual classifiers are exactly what RobustNeRF had to smooth. Our hypothesis is that smaller patches trade wire erasure for leaked distractor fragments, because a single image cannot tell a poorly fit wire from a pedestrian's edge. The patch-size sweep on the synthetic set will settle it. If 4×4 patches match our thin-region scores without losing global PSNR, the 3D voting is unnecessary and we will say so.

Why not just protect wires with a 2D thin-structure or edge detector? Semantic segmenters score near 100% error on the ultra-thin class in DDOS. Edge detectors fire equally on distractor silhouettes. 3DGS-HPC also reports that pretrained-model masks are fragile early in training. We will include an edge-protected variant of 3DGS-HPC as an extra baseline. The prediction is that it keeps more wires but reintroduces distractor floaters along moving object boundaries.

Why not just store residual statistics per Gaussian and skip the grid? This is simpler and would share the time-span idea. Its limitation is that it can only judge primitives that already exist. Under delayed growth, which RobustSplat shows is needed against floaters, wires often have no Gaussians to carry evidence, and evidence is lost whenever primitives are pruned. The per-Gaussian ablation tests this directly. If it matches the full method, the grid adds only engineering, and the contribution reduces to the time-span criterion.

Why not just tune RobustSplat's schedule to densify earlier? Earlier global densification is precisely what spawns transient floaters. The question is whether any schedule sits on a better frontier. We will sweep RobustSplat's growth delay and plot thin-region score against distractor-region floater area. Our method should lie above that curve, not at one point on it.

Proposed Experiments

We need a benchmark that has both thin structures and distractors, and none exists in the file. So we construct one in two ways. First, synthetic scenes built in a Kubric-style renderer, like the synthetic sets used by RobustNeRF and NeRF On-the-go. These contain wires, fences and slender furniture at controlled widths from 0.5 to 4 pixels, plus inserted moving and moved distractors, with exact thin-structure masks and depth. Second, we hand-annotate thin-structure masks on held-out test views of the RobustNeRF benchmark, the NeRF On-the-go dataset, and Phototourism. If camera trajectories can be recovered for DDOS flight sequences, we additionally evaluate depth error on its ultra-thin class from our reconstructions. If they cannot, we report only the synthetic and annotated real results.

Metrics are standard PSNR, SSIM and LPIPS over whole images. We add PSNR restricted to the dilated thin masks, and thin-structure recall: the fraction of ground-truth thin pixels whose rendered depth is within a tolerance. We add floater area in known distractor regions as a check that we have not re-admitted transients. We also report wall-clock training time and memory against 3DGS-HPC's reported 13–17 minutes and about 2 GB.

The decisive check is the synthetic width sweep. If the method is right, the thin-region gap between it and 3DGS-HPC should grow as structures get thinner. Global PSNR and floater area should stay flat. The per-Gaussian and no-time-span ablations should each recover only part of that gap. If instead the 4×4 patch baseline closes the gap, or if floater area rises in proportion to thin recall, the core claim fails.