Efficient Gaussian Splatting for Monocular Dynamic Scene Rendering via Sparse Time-Variant Attribute Modeling

Hanyang Kong, Xingyi Yang, Xinchao Wang

TL;DR

This paper introduces Efficient Dynamic Gaussian Splatting (EDGS), a novel method that significantly speeds up the rendering of dynamic 3D scenes from monocular video. EDGS achieves this by using a sparse representation and selectively modeling only the time-varying attributes of deformable objects, while treating static areas efficiently.

What Problem Does This Paper Address?

Rendering realistic dynamic 3D scenes from monocular video is a challenging task. While recent advancements like 3D Gaussian Splatting (3DGS) have enabled real-time rendering, applying them to dynamic scenes often leads to an explosion in the number of Gaussian primitives needed to capture motion. This redundancy drastically slows down rendering speeds. Furthermore, existing methods unnecessarily re-evaluate attributes for static parts of a scene, which remain constant over time, contributing to inefficiency and potential visual artifacts like jittering. The core problem is how to efficiently represent and render dynamic scenes without sacrificing quality, specifically by reducing the computational burden associated with redundant Gaussians and static scene elements.

Methods

EDGS tackles this problem by introducing a sparse time-variant attribute modeling approach. The key innovations include:

  • Sparse Anchor-Grid Initialization: Instead of dense Gaussians, EDGS uses a sparser grid of "anchor points" derived from Structure-from-Motion (SfM) data. This reduces the initial number of primitives.
  • Time-Mask MLP for Static/Dynamic Separation: A novel, unsupervised "time-mask MLP" is employed to classify each anchor as belonging to either a static or a deformable area of the scene. This is crucial for selective processing.
  • Selective Time-Variant Attribute Modeling: Only anchors identified as part of deformable objects are processed by MLPs to query their time-variant attributes (like location, scale, and quaternion). Static anchors have their attributes (color, opacity) treated as time-invariant and are handled more efficiently.
  • Sparse Motion Flow with RBF Kernel: The movement of deformable Gaussians is modeled sparsely using a classical kernel representation. Specifically, a Radial Basis Function (RBF) kernel is used to compute Gaussian offsets based on anchor features and their relationship to the anchor's overall movement. This avoids dense, per-Gaussian deformation calculations.
  • Densification and Pruning: After initial sparse representation, a densification process similar to standard 3DGS is used to improve detail. A pruning step further refines the representation by removing anchors with low average opacity.

Datasets

The paper evaluates EDGS on two real-world dynamic scene datasets:

  • NeRF-DS (Yan, Li, and Lee 2023): Used for per-scene evaluation.
  • HyperNeRF (Park et al. 2021b): Used for averaged evaluation on its VRIG dataset.

Key Insights and Ideas

  • Static areas are a major source of inefficiency: The paper highlights that modeling static scene elements as if they were dynamic is a significant bottleneck in current methods. EDGS's time-mask MLP effectively addresses this by separating static and dynamic components.
  • Sparse representation is key to speed: By moving from dense Gaussians to a sparse anchor-grid and selectively querying attributes, EDGS drastically reduces the computational load.
  • Unsupervised filtering is powerful: The ability to distinguish static from dynamic regions without explicit masks or supervision is a significant advantage, making the method more generalizable.
  • RBF kernel for semantic motion: Using an RBF kernel for motion flow allows for a more semantically aligned deformation of Gaussians, improving quality for non-rigid objects.

Key Results

  • Significant Speedup: EDGS achieves substantially faster rendering speeds compared to state-of-the-art methods. For example, it reaches 117 FPS on the HyperNeRF dataset, significantly outperforming others.
  • Improved Rendering Quality: Despite the speed improvements, EDGS maintains or even surpasses the rendering quality (PSNR, SSIM) of existing methods, particularly for dynamic objects.
  • Reduced Gaussian Count: The method requires fewer Gaussians to be queried for time-variant attributes, directly contributing to the speedup.
  • Qualitative Improvements: Visualizations show EDGS reconstructs finer details and produces more structured renderings of moving objects, with less jittering in static areas compared to deformable 3DGS and 4DGS.

Limitations and Future Work

  • Limitations: The paper acknowledges that while the anchor-grid strategy is crucial for speed, a densification step is still needed for higher rendering performance. The RBF kernel approach, while effective, is noted as being VRAM-consuming in its similarity calculation if not implemented carefully (though they use a specific implementation that mitigates this).
  • Future Work: The paper suggests that further exploration of anchor-voxel strategies and time-mask MLPs could lead to even greater efficiency and quality improvements. The core idea of sparse, time-variant attribute modeling could be extended to other rendering paradigms.

Summary updated

Abstract

Rendering dynamic scenes from monocular videos is a crucial yet challenging task. The recent deformable Gaussian Splatting has emerged as a robust solution to represent real-world dynamic scenes. However, it often leads to heavily redundant Gaussians, attempting to fit every training view at various time steps, leading to slower rendering speeds. Additionally, the attributes of Gaussians in static areas are time-invariant, making it unnecessary to model every Gaussian, which can cause jittering in static regions. In practice, the primary bottleneck in rendering speed for dynamic scenes is the number of Gaussians. In response, we introduce Efficient Dynamic Gaussian Splatting (EDGS), which represents dynamic scenes via sparse time-variant attribute modeling. Our approach formulates dynamic scenes using a sparse anchor-grid representation, with the motion flow of dense Gaussians calculated via a classical kernel representation. Furthermore, we propose an unsupervised strategy to efficiently filter out anchors corresponding to static areas. Only anchors associated with deformable objects are input into MLPs to query time-variant attributes. Experiments on two real-world datasets demonstrate that our EDGS significantly improves the rendering speed with superior rendering quality compared to previous state-of-the-art methods.
BibTeX
@inproceedings{kong2025efficient,
  author = {Hanyang Kong and Xingyi Yang and Xinchao Wang},
  title = {{Efficient Gaussian Splatting for Monocular Dynamic Scene Rendering via Sparse Time-Variant Attribute Modeling}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2025},
  doi = {10.1609/aaai.v39i4.32460},
}

Loading related content...

Figure