Free-Space Evidence in the Object's Own Frame for Dynamic Gaussian Splatting

When a car drives past a LiDAR-equipped vehicle, each sweep sees only the side of the car facing the sensor. Over a few seconds, though, the car turns, overtakes, or is overtaken, and the sensor sweeps across many of its sides. Every laser ray that grazes past the car without hitting it says something definite: "this part of space, in the car's own coordinate frame, is empty." This proposal argues that this time-integrated free-space evidence, expressed in each moving object's own frame, is the missing constraint for reconstructing dynamic agents in 3D Gaussian Splatting (3DGS). 3DGS represents a scene as millions of small colored, semi-transparent blobs fit to photographs. Today these reconstructions of moving vehicles come out as hollow shells wrapped in semi-transparent "fuzz," or "floaters," because nothing besides pixel color constrains them.

The same evidence also says where a learned shape prior may act. A prior is safe in the parts of the object frame that were never seen as occupied or as empty, and it must stay out of everything else. We propose a three-way partition of each object's canonical space into occupied, verified-free, and never-observed regions, built from all LiDAR sweeps after the box poses have been refined. Blob positions on each object are driven by an energy defined on this partition: an attraction toward occupied space, a barrier out of free space, and a weak completion prior that operates only in the never-observed region. The claim we will test is that gating the prior by observation evidence gives more complete vehicle geometry than either no prior or an ungated prior, with fewer floaters, and without lowering rendering quality on views that were observed.

Where this sits in prior work

The mechanism is adapted from EnerGS. EnerGS partitions space into occupied, free, and unknown regions using LiDAR. It places a robust attraction on occupied space and a Softplus barrier on free space. It lets photometric gradients update color, scale, and opacity but not position, and it prunes blobs that stay stuck in free space. It reports lowering the leak ratio on KITTI from 12.5 to 9.2. EnerGS assumes a static world, and it never fills in unseen surfaces; its unknown region is deliberately left to photometry. Our work moves the partition into per-object frames, accumulates it over time, and gives the unknown region a gated prior.

The idea that motion supplies multiple views of an object comes from TraPCC. TraPCC aggregates LiDAR hits on moving vehicles across frames into pseudo ground truth for a point-completion network and reports a mean Chamfer distance of 0.0973 on KITTI. TraPCC uses only hits, not the free space along rays. It produces points with no appearance. It fails below a sparsity floor and is corrupted by misalignment during aggregation.

The use of free-space ray consistency next to a learned shape prior echoes AML. AML fits a shape prior over signed distance fields, learned from synthetic CAD models, to real laser scans and reports 9.1 cm completeness error on KITTI. AML applies its prior everywhere, to segmented single objects, with no radiance model. Our design applies the prior only where evidence is absent. Latent Uncertainty-Aware Multi-View SDF Scan Completion supports the gating idea from a different angle: it weights partial signed-distance fusion by observability and shows that removing this uncertainty guidance hurts.

The transient-filtering methods RobustSplat and 3DGS-HPC show why a complementary approach is needed. RobustSplat delays densification and bootstraps masks of moving content; 3DGS-HPC classifies moving content patch by patch. Both treat moving objects as nuisances to delete, and 3DGS-HPC explicitly leaves the regions those objects cover as holes. Split4D does reconstruct moving foreground, but it leaves occlusion holes and blurry contact boundaries. We target exactly the geometry these methods discard or leave incomplete.

Core Claim

Each tracked rigid agent $i$ has a canonical frame. Every LiDAR ray at time $t$ is transformed into that frame using the refined box pose. Each voxel accumulates hit counts $h$ and pass-through counts $f$. A voxel is labeled occupied if $h \geq k$, verified-free if $f \geq k$ and $h = 0$, and never-observed otherwise. Here $k$ is a small consistency threshold that absorbs pose jitter. The positional energy for a blob centered at $\mu$ is

$$E(\mu) = \rho\big(d_{occ}(\mu)\big)\,\mathbb{1}_{occ} + \mathrm{softplus}\big(\beta\, s_{free}(\mu)\big)\,\mathbb{1}_{free} + \lambda\,\big|\phi_{prior}(\mu)\big|\,\mathbb{1}_{unk}.$$

In this energy, $\rho$ is the Welsch robust penalty on the distance $d_{occ}$ to the nearest occupied evidence. The term $s_{free}$ is the signed penetration depth into verified-free space, and $\beta$ is the barrier sharpness. The function $\phi_{prior}$ is a signed distance predicted by a completion network conditioned on the occupied evidence, and $\lambda$ is a small weight. The indicators select the region containing $\mu$. Static background keeps the original EnerGS energy, built from LiDAR returns with the in-box points removed.

The key observation is that, for a rigid agent, verified-free space grows with the range of viewing angles covered during the trajectory. The never-observed region therefore shrinks to the parts that genuinely no ray ever reached, typically the underside and the far side of cars seen from only one lane. In those parts, and only there, the prior shapes the geometry.

Research Program

We will build the system on a box-anchored dynamic 3DGS, with one blob set per tracked vehicle and one static set. Positions are decoupled from photometric gradients as in EnerGS. Box poses are first refined by registering the aggregated hits, then fine-tuned with the photometric loss. This step addresses TraPCC's misalignment failure.

The completion network has two candidate sources: an AML-style prior over signed distance fields trained on synthetic car models, and a network trained on TraPCC-style pooled real scans. Comparing the two directly tests the gap between synthetic and real data under gating.

We will also analyze where gating matters. We will measure how the size of the never-observed region shrinks with trajectory angular coverage and with distance from the sensor. We will then plot the gain from completion against that coverage. If the method works, the gain should be largest at intermediate coverage. With near-total coverage nothing needs filling. With near-zero coverage we expect the prior to collapse, because the sparsity floor described by TraPCC applies to us as well.

Baselines

The naive reference is the same box-anchored dynamic 3DGS with ordinary photometric optimization and densification. It is the configuration that produces hollow, fuzzy vehicles, and every other variant is measured against it.

The strongest competitor is EnerGS applied to the static background, with LiDAR points inside tracked boxes removed, combined with plain photometric dynamic objects. This is what a careful practitioner would build first. EnerGS reports 24.81 PSNR on Waymo, 16.80 PSNR on KITTI, and a KITTI leak ratio falling from 12.5 to 9.2. We will rerun it on our scene splits, with the same blob budget, iteration count, and hyperparameter search budget as our method, rather than quoting those numbers as directly comparable. Beating it means lower Chamfer distance on held-out vehicle geometry and a lower in-box leak ratio, with PSNR on held-out frames within noise of the baseline.

A second competitor adds TraPCC-style aggregated hits as a depth and attraction term on the object blobs. This baseline uses the same multi-frame idea without free space and without a prior.

Ablations isolate each ingredient: the object-frame barrier removed, the prior applied everywhere instead of gated, and the prior removed entirely. The comparison of the gated and ungated prior is the decisive one for the core claim.

Why not just…?

Why not just mask dynamic LiDAR points with boxes and run EnerGS? This fixes EnerGS's static-world violation for the background, and we expect it to be competitive there. We therefore make it the main baseline rather than arguing against it. It gives the vehicles no geometric constraint at all, so floaters and hollow shells on vehicles remain. The comparison on in-box leak ratio and held-out vehicle Chamfer distance settles whether anything beyond this baseline is needed.

Why not just accumulate points over time like TraPCC and supervise depth? Hits constrain only surfaces that were seen. They say nothing about the empty space around the object, where floaters live. TraPCC also reports failure under sparse and misaligned aggregation. The aggregated-hit baseline above measures how much free space and gating add on top of hits.

Why not just insert a completed shape from PCN or AML as initialization? An ungated prior overrides real observations whenever it disagrees with them. This is exactly the hallucination failure in the sparse regime, and it is made worse by the domain gap between synthetic and real data. Our ungated-prior ablation is this alternative, evaluated under the same budget.

Why not just turn off densification on moving objects? RobustSplat reports that disabling densification yields overly smooth reconstructions lacking detail. It also does nothing to fill in unseen sides.

Proposed Experiments

We use the Waymo Open Dataset and KITTI Tracking, both of which provide boxes and LiDAR.

The decisive test is a temporal holdout. For each vehicle, we train on the first part of its trajectory and evaluate the Chamfer distance to its LiDAR hits from later frames. These later frames include sides never seen during training, which gives real ground truth for completion without synthetic data. We report results stratified by the angular coverage of the training segment.

Floaters are measured with the EnerGS leak ratio, computed inside the boxes against the full-sequence free space. Appearance is measured with PSNR, SSIM, and LPIPS on held-out camera frames, cropped to vehicles and also on full frames.

A second check uses vehicles that are briefly occluded by other moving agents. It tests whether pass-through counts survive occlusion without falsely carving the occluded object.

The hypothesis fails if the gated prior does not beat both the no-prior and the ungated-prior variants on held-out Chamfer distance at intermediate coverage. It also fails if the object-frame barrier lowers PSNR on observed views. Either outcome would mean that time-integrated free-space evidence does not supply the constraint we claim.