Robots that build 3D maps independently, or one robot that returns to a room a day later, need to stitch those maps into one coordinate frame without being told roughly how they line up. Recent systems do this well for 3D Gaussian splatting maps. These maps represent a scene as millions of small, semi-transparent colored ellipsoids that can be rendered into images very quickly. The catch is that every such system assumes nothing in the scene moved between the two captures. Real rooms violate this all the time: chairs get pushed, people walk through, boxes appear and disappear. When that happens, alignment methods can lock onto the furniture that moved rather than the walls that did not, and they end up confidently wrong.
We propose to register two Gaussian maps and explain what changed between them in a single step. Each Gaussian carries a compact visual-semantic feature lifted from a 2D foundation model. A matching procedure in the style of optimal transport, one that is allowed to leave mass unmatched, decides jointly which Gaussians correspond, which transform aligns the static background, and which leftover groups of Gaussians moved rigidly, appeared, or vanished. The two-sentence version: today's initialization-free splat registration treats change as noise to be outvoted, and this fails once moved objects are numerous or look alike. We treat change as structure. Unmatched mass identifies new or removed content, and a second rigid fit on that leftover mass recovers moved objects, so the background transform is no longer contaminated by them.
Core Claim
The nearest existing work is SIREN. It distills CLIP features into Gaussians, matches Gaussians by semantic similarity, and recovers scale, rotation and translation with weighted SVD inside RANSAC, then refines with rendered views. It reports errors tens to hundreds of times smaller than GaussReg and PhotoReg. Its gap is stated in the file: it works entirely under the static-scene assumption. RegGS aligns Gaussian sub-scenes as mixtures using a differentiable Sinkhorn approximation of the 2-Wasserstein distance. Its balanced transport has the same assumption built in, because every unit of mass in one map must go somewhere in the other.
Our claim is specific. When a substantial and semantically repetitive fraction of a scene has changed, inlier-count RANSAC and balanced transport both fail in predictable ways. RANSAC can select the moved-object hypothesis. Balanced transport drags the background toward displaced mass. Unbalanced transport over geometry plus semantic features fixes both, provided hypotheses are scored by how much spatially spread-out mass they explain rather than by how many matches they collect. As a byproduct the method returns per-object rigid motions. Distractor filters such as RobustSplat and 3DGS-HPC simply throw that information away.
Research Program
Each Gaussian $i$ in map $A$ has a mean, covariance, mass $a_i$ (opacity times projected volume), and a feature $f_i$. The feature comes from DINOv2 features distilled during map building, compressed to about 16 dimensions with PCA. Keeping it this small targets the 40–50% distillation overhead SIREN reports, and we will measure whether 16 dimensions suffices. Map $B$ is defined the same way, with masses $b_j$.
For a candidate similarity transform $T$, the matching cost $C_{ij}(T)$ adds two terms. The first is the closed-form 2-Wasserstein distance between Gaussian $i$ transformed by $T$ and Gaussian $j$, reusing RegGS's mixture view. The second is $\lambda\lVert f_i-f_j\rVert^2$, where $\lambda$ weights the feature distance. We solve
$$\min_{P\ge 0}\ \langle C(T),P\rangle-\varepsilon H(P)+\tau\,\mathrm{KL}(P\mathbf{1}\,\|\,a)+\tau\,\mathrm{KL}(P^{\top}\mathbf{1}\,\|\,b),$$
where $P$ is the soft correspondence matrix, $H$ is its entropy with weight $\varepsilon$, and $\tau$ sets how expensive it is to leave mass unmatched. Smaller $\tau$ lets more mass stay unmatched. The standard unbalanced Sinkhorn iterations solve this on the GPU. A Gaussian whose transported mass $(P\mathbf{1})_i/a_i$ is low becomes a candidate for change. Because the KL penalty degrades smoothly, masking out transient regions no longer produces the abrupt mass gaps that the research file flags as a hazard for Sinkhorn (its Gap 4).
The pipeline has three stages. First, global hypotheses come from feature-matched Gaussian triplets, as in SIREN, so no initial pose is needed. Each hypothesis is scored by the transported mass under unbalanced transport, weighted by the spatial spread of the matched mass, which we measure as the volume of the bounding hull. A tight cluster of ten identical chairs that moved together therefore cannot outscore walls and floor. Second, we refine the best hypothesis by gradient descent on the objective above. Third, we cluster the unmatched mass by feature and spatial proximity and run a second, local hypothesis-and-refine pass on each cluster. Clusters that align under their own rigid transform are labelled moved. The rest are labelled appeared or removed. A final photometric refinement, like SIREN's, renders only the Gaussians labelled persistent.
Baselines
The naive reference is GaussReg. It extracts opacity-filtered Gaussian centers, aligns them with a geometric transformer, and refines with multi-view stereo on rendered images. It reports a 100% success ratio on ScanNet-GSReg in about 4.8 s. It is the right "do nothing about change" reference, and on unchanged scenes we must match it.
The strongest competitor is SIREN, because it already solves initialization-free registration with semantic features. That leaves dynamics as the only open axis, which is exactly our question. The file gives only relative improvements for SIREN, not absolute errors on any benchmark, and no reported result found on Lacuna for SIREN on any changed-scene setting. We will therefore re-run it ourselves with its own CLIP distillation, its RANSAC budget, and its refinement stage.
RegGS is the balanced-transport competitor. We will run it with identical Gaussian inputs. No reported numeric ATE is in the file.
All methods receive the same reconstructed maps, the same GPU, and the same wall-clock budget. Hyperparameters are tuned on a held-out split of unchanged scenes only, so that no method is tuned on change. Beating SIREN means two things: a higher registration success rate (rotation under 5° and translation under 5% of scene extent) once the changed-mass fraction exceeds a moderate level, and no loss in accuracy on unchanged scenes.
Three ablations isolate the ingredients. The first replaces unbalanced transport with balanced transport. The second removes features from the cost. The third replaces spread-weighted scoring with plain inlier counting. This last ablation is our key ingredient, and we expect it to carry most of the gain.
Why not just…?
Why not just run SIREN, since RANSAC already rejects outliers? This is the real threat. If moved objects make up a small, semantically unique fraction of the scene, RANSAC will likely outvote them, and SIREN will match us on alignment. Our hypothesis is narrower: failure appears when moved objects are numerous and look alike, such as rows of chairs. Then the moved cluster produces a coherent, high-inlier competing hypothesis. The changed-fraction sweep below settles this directly. If SIREN never breaks, the proposal's contribution shrinks to recovering object motion, and we will report it that way.
Why not just mask movable classes with a 2D segmenter before building maps? Furniture is the main thing that moves between visits, and it cannot be masked out wholesale without deleting much of the static structure needed for alignment. 3DGS-HPC also shows that category-driven masking misses uncatalogued changes such as shadows and props. We will include "SIREN plus masking of person and furniture classes" as a baseline. Masking also gives up the moved-object output entirely.
Why not just use balanced optimal transport, as RegGS does, with a trimmed loss? Trimming the worst-matched fraction, as in RobustNeRF's 50% trimmed loss, needs a fixed inlier ratio that is unknown across visits. Balanced transport still assigns transient mass to wrong partners before trimming happens. The balanced-versus-unbalanced ablation, run with a tuned trimming ratio, tests this directly.
Proposed Experiments
ScanNet has instance annotations (ScanNet). We will use them to build a changed version of ScanNet-GSReg. In map $B$ we move annotated instances rigidly, delete some, and insert copies of others, sweeping the changed-mass fraction from 0% to about 50%. We also separately control how many moved instances share a category. Ground truth includes both the background transform and each object's motion.
For real dynamics, we build two submaps from disjoint time windows of BONN RGB-D Dynamic sequences. People occupy different places in each window, and the ground-truth trajectories give the true relative pose. Splitting NeRF On-the-go and RobustNeRF captures into halves gives transient-heavy pairs whose relative pose comes from their SfM poses.
Metrics are rotation, translation and scale error, success rate, moved-object pose error, and per-Gaussian change-labelling F1 on the synthetic set. We also report PSNR of the fused map rendered at held-out views, and runtime.
The decisive check is the success-rate curve against changed fraction for SIREN, RegGS, and our method with and without spread-weighted scoring. The proposal holds only if SIREN's curve falls where ours stays flat, and if the gap is concentrated in the conditions with many same-category moved objects. A second check is a 0% change control, where we must stay within noise of GaussReg's and SIREN's accuracy. A third check is whether 16-dimensional features keep these gains at no more than half of SIREN's distillation cost.
References
- SIREN: Semantic, Initialization-Free Registration of Multi-Robot Gaussian Splatting Maps
- RegGS: Unposed Sparse Views Gaussian Splatting with 3DGS Registration
- RobustSplat: Decoupling Densification and Dynamics for Transient-Free 3DGS
- 3DGS-HPC: Distractor-free 3D Gaussian Splatting with Hybrid Patch-wise Classification
- GaussReg: Fast 3D Registration with Gaussian Splatting
- RobustNeRF: Ignoring Distractors with Robust Losses
- ScanNet: Richly-Annotated 3D Reconstructions of Indoor Scenes