Systems that render moving scenes in real time spend much of each frame moving primitives they did not need to move. In dynamic 3D Gaussian Splatting, a scene is a cloud of small colored blobs ("Gaussians") that are projected and blended into an image. Before every frame, current methods run a small neural network or evaluate a per-blob trajectory for every blob, even though most of a real scene is a static wall, floor or table, and most of the moving part is a handful of rigid limbs. We propose to make the cost of motion scale with the number of distinct motions in the scene rather than with the number of blobs.
The representation is a single kinematic scene tree. A kinematic tree is a hierarchy in which each rigid part moves relative to its parent, the way a forearm moves relative to the upper arm. In our version, the root is the static world, every articulated body hangs off it as a subtree of rigid parts, and only a small, explicitly budgeted set of blobs gets free-form per-frame deformation. The same structure does two jobs. It cuts per-frame compute, because a frame needs one rigid transform per tree node, a cost that does not grow with blob count. It also constrains reconstruction from a single moving camera, because all blobs on a rigid part must share one motion and therefore cannot slide independently along camera rays to fake the right pixels.
Core Claim
Our claim is that, on real dynamic captures, a learned assignment of each Gaussian to one of three motion tiers matches the image quality of per-Gaussian deformation methods while cutting deformation work before rasterization by a large factor. The tiers are static, kinematic (attached to a node of a discovered tree), and residual (free deformation under a compute budget). We further claim that the kinematic tier is what makes this possible. A version that only splits static from dynamic, or that clusters dynamic blobs into rigid groups without parent-child chaining, should either lose quality or need far more residual blobs to recover it. A third claim is a side effect: forcing transient blobs to either join a moving tree node or be removed should reduce the mid-air "floaters" that come from treating dynamic content as static.
The per-frame cost we optimize is explicit. Let $N_n$ be the number of tree nodes, $N_r$ the number of residual Gaussians, $N$ the total number of Gaussians, and $c_{fk}$, $c_{res}$, $c_{rast}$ the per-unit costs of composing one rigid transform, evaluating one residual deformation, and rasterizing one Gaussian. Then
$$C_{\text{frame}} = N_n c_{fk} + N_r c_{res} + N c_{rast},$$
whereas a deformation-field method such as 4D-GS pays roughly $N(c_{res} + c_{rast})$. Training minimizes
$$\mathcal{L} = \mathcal{L}_{\text{photo}} + \lambda\, \mathbb{E}[C_{\text{frame}}],$$
where $\mathcal{L}_{\text{photo}}$ is the usual image reconstruction loss. The expectation is taken over soft tier assignments that are annealed to hard ones. Sweeping $\lambda$ traces a curve of quality against cost, so the method can be compared at matched quality or at matched cost rather than at one cherry-picked point.
Relation to Prior Work
The closest work is Template-free Articulated Gaussian Splatting. It groups Gaussians into rigid "superpoints," finds joints where two superpoints keep a constant distance while rotating relative to each other, links them with a minimum spanning tree, and skins Gaussians to the result. It reports over 110 FPS and 39.23 dB PSNR on D-NeRF. We reuse its joint-discovery step. That paper, however, treats a single isolated object, explicitly does not separate dynamic foreground from background, and uses the skeleton to support re-posing rather than to manage compute. We extend discovery to a forest rooted at the static world, allow several bodies, and add the residual tier with a cost penalty, which is what lets the method handle scenes that are not purely articulated.
TreeSplat is the strongest constraint on our novelty and speed claims. It organizes per-Gaussian motion coefficients in a tree built during densification, sums them along root-to-leaf paths, and then merges the tree into one flat coefficient vector per Gaussian. It reports 230 FPS on D-NeRF and 206 FPS on N3V at 37.11 dB on D-NeRF. Its tree is an abstract clustering with no bone lengths, no shared rigid transforms, and no re-posing. After merging, every Gaussian still evaluates its own time-basis trajectory each frame, so its deformation cost remains proportional to $N$. Our bet is that sharing one rigid transform across thousands of blobs beats per-blob trajectories as scenes grow, and that the rigid sharing helps in the single-camera setting where TreeSplat's per-blob freedom is a liability.
4D-GS supplies the standard per-Gaussian deformation field: six 2D feature planes queried and decoded by a shallow network. It reports 82 FPS at 800×800 on an RTX 3090 and 34.05 dB on D-NeRF, and it has no kinematic prior. 1000+ FPS 4D Gaussian Splatting attacks latency from a different side, by pruning short-lived flickering Gaussians and sharing visibility masks across neighboring frames. That approach is complementary: it reduces $N$, while we reduce what each frame does per Gaussian. Its exact headline number is truncated in our file, so we report no figure for it. Learnable Infinite Taylor Gaussian also ties dense local Gaussians to sparse global ones with blend skinning, but its anchors are unconnected and its authors state that computational demands remain high. It motivates the view that hierarchy helps quality but has not yet been used to buy speed.
On the static-scene side, RobustSplat shows that densification misreads moving content as missing static detail and spawns floaters. Its fix is to delay Gaussian growth and bootstrap masks of transient content. We adopt its delayed growth. Rather than discarding transient content, we route it into a moving tier. Split4D separates static from dynamic Gaussians using learned identity features and velocities, without a video tracker. It notes that Gaussians get shared across identities under tight contact, a failure we expect to inherit. HuGS shows that forward skinning with a shallow residual network renders humans at about 80 FPS at 512×512 on a V100 and reaches 32.49 dB on THuman4, but it needs a pre-tracked skeleton. Our discovered tree removes that requirement at the price of the motion-dependence noted below. Finally, the pose-estimation literature constrains the design. Root-to-leaf parameterization compounds errors toward the end-effectors, as discussed around bone-length and direction decompositions such as Jingwei Xu et al.. We therefore give each node a small learned correction and measure end-effector error separately.
Research Program
The system is built in three stages on a standard Gaussian Splatting rasterizer. The first stage is a warm-up. A deformation-field model in the style of 4D-GS is trained briefly, with RobustSplat-style delayed densification, so that early growth is not fooled by moving content. The second stage builds the structure. Gaussians are clustered into superpoints by trajectory similarity. Clusters whose motion is near zero are attached directly to the world root. Joints among the moving clusters are found with the constant-distance criterion from the template-free method, and the clusters are assembled into a forest whose roots attach to the world. In the third stage, the model is retrained with soft tier assignments under the cost-penalized loss above. Kinematic Gaussians are skinned to at most four nearby nodes, and residual Gaussians use a small deformation network. Superpoints are split or merged during this stage, as in the template-free method.
At inference, assignments are hard. Static Gaussians are transformed once per camera, kinematic Gaussians apply precomputed per-node transforms obtained by composing transforms down the forest, and only residual Gaussians run a network. Because forward kinematics touches $N_n$ nodes, and $N_n$ is in the hundreds, the step before rasterization should shrink to near the cost of skinning alone.
The analysis studies three questions. The first is how many distinct motions real benchmark scenes actually contain, which we measure by the number of nodes and the residual fraction at matched PSNR. The second is whether kinematic sharing reduces depth-velocity artifacts in single-camera video. The third is where the approach fails. We expect it to fail on fluid, cloth-dominated or topology-changing content, where the residual tier should grow and the speed advantage should vanish. We will report this outcome rather than hide it.
Baselines
The naive reference point is 4D-GS run as released, since every dynamic Gaussian method in our file measures against it. On D-NeRF, it reports 34.05 dB PSNR and 82 FPS at 800×800 on an RTX 3090 (4D-GS). We rerun it on our hardware and report both its published and our measured numbers.
The strongest speed competitor is TreeSplat, with 230 FPS on D-NeRF, 206 FPS on N3V, and 37.11 dB PSNR on D-NeRF (TreeSplat). It is the strongest because it already uses a hierarchy to stabilize training and then removes the hierarchy's runtime cost. The strongest quality competitor on articulated content is the template-free articulated method, at 39.23 dB on D-NeRF and over 110 FPS (Template-free Articulated GS). For 4DGS-1K, no complete reported result was found on Lacuna, because the file is truncated. We run its released code if available and otherwise drop it.
For fairness, all methods share the same training and test splits, resolution, and GPU. They are trained for their published iteration counts, and we use the same tuning budget for the key hyperparameters of each method, which is our $\lambda$ and the residual network size. We also report a matched-Gaussian-count setting so that speed gains cannot come merely from a smaller model. A method beats TreeSplat if, at PSNR within 0.2 dB of it on N3V, it renders faster with lower measured time spent before rasterization. A second criterion, at matched FPS on single-camera sequences, is higher PSNR and lower LPIPS. A tie on D-NeRF alone would not count, because those scenes are small and mostly articulated.
Two ablations isolate the key ingredient. The first is "static plus free dynamic." It assigns Gaussians only to static or residual tiers using Split4D-style velocity separation and has no kinematic tier. It tests whether the static-world split alone explains the speedup. The second is "flat rigid clusters." It keeps rigid superpoints but removes parent-child chaining and bone-length consistency. It tests whether the tree, rather than mere rigid grouping, matters for quality, end-effector error, and single-camera stability.
Why not just…?
Why not just mask out the static background and run 4D-GS on the foreground? This is a real threat for speed. If most Gaussians are static, skipping their deformation already captures much of the gain. The "static plus free dynamic" ablation is exactly this baseline. Our claim survives only if the foreground itself is expensive enough that sharing transforms within it matters, and only if rigid sharing improves single-camera quality where free foreground deformation is known to slide blobs along rays. If the ablation matches us on both speed and quality, the kinematic tier is not worth its complexity, and we will say so.
Why not just use TreeSplat, which is already over 200 FPS? TreeSplat's merged coefficients still cost a per-Gaussian trajectory evaluation per frame, so its deformation cost grows with scene size. Its tree also carries no rigid structure that could constrain single-camera ambiguity or support re-posing. We test this directly with a scaling experiment. We grow the static background by reconstructing wider N3V crops at higher Gaussian counts and plot pre-rasterization time against $N$ for both methods. If TreeSplat's curve stays flat in practice because trajectory evaluation is negligible next to rasterization, our speed claim reduces to the quality and re-posing claims, and we will present it that way.
Why not just prune harder, as 4DGS-1K does? Pruning reduces $N$ and is compatible with our method, not an alternative to it. We will apply the same spatial and temporal pruning on top of our method and of the baselines. If the gap closes once everyone prunes, then the deformation step was never the bottleneck on these benchmarks.
Proposed Experiments
The main experiments use D-NeRF for single-camera synthetic articulated scenes, N3V/Neu3D for real multi-view scenes with large static backgrounds, HyperNeRF for real single-camera deformable scenes that are poorly articulated, and ZJU-MoCap and the Robots dataset from the template-free work for articulated bodies with held-out poses. We report PSNR, SSIM and LPIPS. We report FPS together with a timing breakdown of the step before rasterization against rasterization itself, along with the residual fraction, the node count, and the variance of bone lengths over time.
The decisive checks are four. The first is the quality-against-cost curve over $\lambda$ on N3V, against TreeSplat and 4D-GS at matched quality. The second is the background-scaling plot described above. The third is a single-camera stress test on HyperNeRF and D-NeRF. Here we measure geometric distortion via depth error against the multi-view reconstruction where it is available, and compare our method with the flat-cluster ablation. This tests whether chaining reduces ray-direction sliding. The fourth is a floater count on N3V. We render static-only views of frames after people leave and compare against 4D-GS and RobustSplat-style filtering, to test whether routing transients into moving nodes removes leftover blobs.
We expect HyperNeRF to be the honest failure case. If the residual fraction there exceeds roughly half of all Gaussians, the method should simply fall back toward 4D-GS cost. Showing that clean fallback is part of the result.
References
- Template-free Articulated Gaussian Splatting for Real-time Reposable Dynamic View Synthesis
- TreeSplat: Mergeable Tree for Deformable Gaussian Splatting
- 4D Gaussian Splatting for Real-Time Dynamic Scene Rendering
- 1000+ FPS 4D Gaussian Splatting for Dynamic Scene Rendering
- Learnable Infinite Taylor Gaussian for Dynamic View Rendering
- RobustSplat: Decoupling Densification and Dynamics for Transient-Free 3DGS
- Split4D: Decomposed 4D Scene Reconstruction Without Video Segmentation
- Human Gaussian Splatting: Real-Time Rendering of Animatable Avatars
- Deep Kinematics Analysis for Monocular 3D Human Pose Estimation