Time-Persistent Geodesics for Separating Touching Objects in Dynamic Gaussian Scenes

Current methods that rebuild a moving scene as a cloud of small 3D blobs (Gaussian splatting) decide which blobs belong together by how close they are in space, or by how similarly they move. Both cues fail at the moment that matters most: when a hand grasps a mug, two people shake hands, or two animals rub against each other. At that instant the touching surfaces are millimetres apart and move together, so blobs on both sides get merged. The result is blurred object boundaries and parts that are forced to move as one rigid piece. There is a better signal available. Distance measured along the surface (geodesic distance) between a fingertip and the mug it touches is large, because the surface path must go around the gap. A dynamic scene also shows those two surfaces apart at other moments in the video.

We propose to build the surface connectivity graph of a dynamic Gaussian scene from the whole trajectory. Two neighbouring blobs stay linked only if they remain close throughout the sequence and their local surfaces agree. Shortest paths on this graph then define a time-persistent geodesic distance. That distance replaces Euclidean proximity in two places: when blobs are grouped into jointly moving clusters, and when per-blob identity features are clustered into objects. The claim is that contact-induced merging, which is the documented failure of the leading decomposition method, can be removed at close to the cost of a nearest-neighbour graph. The only prior dynamic use of geodesics needed about fifteen minutes per frame.

Core Claim

Static geodesic segmentation already shows that surface distance stops labels leaking across narrow gaps. Geodesic-Former embeds Dijkstra shortest-path distances on a point graph as positional encodings and raises 1-shot ScanNetV2 instance segmentation from 6.2 to 10.6 mAP. GSTran suppresses attention across points that leave a neighbour's tangent plane. Both methods operate on a single frame, and a single frame is exactly where geodesics are weakest: noise points or contact can create bridge edges across a gap, and a bridge edge short-circuits the geodesic.

Our observation is that time breaks the tie. A spurious bridge between a finger and a mug exists only during contact, while a true surface edge persists. We therefore define edges by their worst case over time. The resulting geodesic stays large across a contact that was ever released, even while the contact holds.

For contacts that never release, time gives no signal. There the edge rule falls back to a surface-normal disagreement test. We state openly that the two regimes behave differently and evaluate them separately.

Method

Each Gaussian $i$ has a center trajectory $\mu_i(t)$ over sequence times $t \in \{t_1,\dots,t_T\}$ and a unit normal $n_i(t)$, taken from its smallest-scale axis. Split4D uses linear velocities, so evaluating trajectories at $T$ times costs little.

We start from a $k$-nearest-neighbour graph in canonical space. An edge between $i$ and $j$ survives only if

$$ w_{ij} = \max_{t}\ \Big( \|\mu_i(t)-\mu_j(t)\| + \lambda\,\big(1-|n_i(t)^\top n_j(t)|\big) \Big) < \tau, $$

where $\tau$ is a length threshold tied to local Gaussian density and $\lambda$ weights normal disagreement. Surviving edges carry weight $w_{ij}$. The time-persistent geodesic $d_G(i,j)$ is the shortest-path length on this graph, with infinity between disconnected components.

We compute $d_G$ only from a few hundred anchors, using GPU multi-source shortest paths, and refresh the graph every few thousand optimisation steps. This mirrors the anchor-based design of Geodesic-Former rather than computing all pairs.

The geodesic enters two existing systems. In Superpoint Gaussian Splatting (SP-GS), which groups Gaussians into rigidly moving clusters through a learnable association conditioned on spatial proximity, we replace that proximity with $\exp(-d_G(i,c_m)^2/\sigma^2)$ for cluster center $c_m$. A cluster can then no longer span two touching parts.

In Split4D, which learns per-Gaussian identity features by contrastive clustering against per-frame 2D masks, we add a smoothness term that pulls features together only between geodesic neighbours. We also add a repulsion term between pairs that are close in Euclidean distance but far geodesically.

Neither change alters rendering, so view-synthesis speed is preserved up to the refresh cost.

Research Program

The work has three parts.

The first part builds and profiles the graph construction on standard dynamic scenes. We measure how often true surface edges are wrongly cut in sparse regions, where the file notes that geodesics break down, and how often bridges survive. These error rates are measured against ground-truth instance labels.

The second part integrates the geodesic into SP-GS and Split4D and evaluates decomposition on contact-heavy footage.

The third part characterises when the method cannot work. This covers contacts that persist for the whole sequence, and cases where noise creates persistent bridges. This part matters because it tells users when to trust the separation.

We position the work against three kinds of prior art. NON-RIGID MULTI-BODY TRACKING IN RGBD STREAMS is the closest precedent and supplies the core intuition that geodesic propagation unsticks touching bodies. It does so on raw per-frame points with heuristic splitting at roughly 15 minutes per frame, and produces no radiance model. Our method differs in using trajectory-level edges inside an optimised Gaussian representation.

Split4D motivates the work: its authors report shared Gaussians and boundary blurring under "extremely tight interactions."

Geodesic-HOF and ClusteringSDF show that surface-tied metrics prevent corner-cutting and label bleeding. Both are static-only, and we borrow the principle but not their implicit machinery.

Baselines

The naive reference is Split4D run unchanged. Its reported number is the anchor: 0.893 mIoU on the Multi-Human dataset, against 0.592 for the tracking-based SA4D, both reported in Split4D. Split4D is the strongest competitor because it already abandons fragile 2D video tracking and uses motion as identity, which is precisely the cue our method claims is insufficient at contact. We run it with the authors' settings, the same per-frame masks and the same iteration count, and give our variant no extra tuning budget beyond sweeping $\tau$ and $\lambda$ on one held-out scene. Beating it means a clear gain in mIoU measured inside contact regions, without losing overall mIoU.

For motion grouping, the baseline is SP-GS with its Euclidean association, trained on identical data and schedules. Beating it means lower rendering error around articulations and contacts at comparable frames per second. SP-GS reports up to 227 FPS on D-NeRF on a V100. We found no reported PSNR for SP-GS on Lacuna, so we will rerun it ourselves.

Two ablations isolate the key ingredient. The first builds edges from a single canonical frame instead of the maximum over time. This single-frame geodesic in the style of Geodesic-Former tests whether time persistence is what matters. The second uses the time-maximum pairwise distance directly as a clustering affinity, with no shortest paths. This tests whether propagation along the surface adds anything beyond the edge criterion.

Why not just…?

Why not just use better 2D masks from video segmentation models? Split4D already shows that tracking-dependent lifting (SA4D, 0.592 mIoU) trails track-free decomposition (0.893). In Object and Contact Point Tracking in Demonstrations Using 3D Gaussian Splatting, chaining SAM 2, RAFT and FoundationPose reaches only 17% high-accuracy tracking success, and it fails precisely during hand-object contact. Masks at contact are the unreliable input, so a 3D prior that does not depend on them is complementary. We will still feed our method and Split4D the same strongest available masks, so that mask quality is not a confound.

Why not just cluster on trajectory distance, since touching objects usually separate at some point? This is the most serious threat. The max-over-time affinity ablation tests it directly. If shortest paths add nothing over thresholded trajectory affinities, the simpler method should be preferred, and we will report that outcome. We expect propagation to matter for large non-rigid objects. On a person, distant body parts are rarely within $\tau$ of each other at all, so only chains of surface edges can connect them into one object without also connecting them to a touching neighbour.

Why not just use more, smaller superpoints so that no cluster spans a contact? Shrinking clusters does reduce merging. It also weakens the as-rigid-as-possible regularisation that makes SP-GS stable, and it raises deformation cost. We will sweep the cluster count $M$ for Euclidean SP-GS and compare the best point on its quality-speed curve against geodesic SP-GS at fixed $M$.

Proposed Experiments

Decomposition is evaluated on Multi-Human, Neural3DV and SelfCap, the datasets on which Split4D reports results. Besides overall mIoU, we report contact-band mIoU: we locate Gaussians of different ground-truth instances within a few centimetres of each other, project them, and score only pixels within that band. We further split frames into transient-contact and whole-sequence-contact cases. We predict clear gains in the transient case and gains that come only from the normal term in the persistent case.

Motion grouping is evaluated on D-NeRF, HyperNeRF and NeRF-DS with PSNR, SSIM and LPIPS, plus FPS. We found no reported quality number for SP-GS on Lacuna, so all baselines on these datasets will be rerun ourselves.

A controlled stress test on scenes where we script contact duration lets us plot separation accuracy against the fraction of the sequence spent in contact.

The decisive check is that the time-maximum edge rule beats the single-frame geodesic ablation inside contact bands on Multi-Human. If it does not, the central claim is false.

Graph refresh time is reported per frame against the 15-minute-per-frame figure of the prior geodesic splitting system. The target is seconds per refresh on one GPU.