ATGS: Anchored Temporal Gaussian Splatting for Long Volumetric Video Representation

Jiahao Wu, Jie Liang, Die Hu, Jiayu Yang, Kaiqiang Xiong, Xiang Li, Xiaoyun Zheng, Chao Wang, Ronggang Wang

TL;DR

ATGS (Anchored Temporal Gaussian Splatting) introduces a novel framework for reconstructing high-quality volumetric videos of long sequences with complex, fast-moving actions. By organizing 3D Gaussians around time-conditioned anchors and employing a temporal windowing strategy, it successfully scales volumetric video representation to thousands of frames in a single training run without suffering from motion drift or severe visual artifacts.

What Problem Does This Paper Address?

Volumetric video enables immersive, free-viewpoint rendering of dynamic real-world scenes, which is crucial for applications like VR/AR, gaming, and sports broadcasting. While recent advancements in 3D Gaussian Splatting (3DGS) have made real-time, high-fidelity static rendering possible, extending this to the temporal domain remains a massive challenge.

Existing dynamic neural rendering methods face a double whammy: long-sequence reconstruction (minute-level videos) and complex motion modeling (fast, multi-person movements). Deformation-based approaches try to warp a canonical space over time, but they struggle to track long-term trajectories and suffer from accumulated optimization errors. Conversely, state-of-the-art 4D Gaussian methods are often constrained to short clips of just a few seconds (20 to 300 frames) and fail to scale, leading to temporal instability, blurriness, and missing geometry over extended durations.

Methods

Instead of explicitly tracking the long-term trajectories of individual Gaussian primitives—an inherently unstable process—ATGS decomposes long-term dynamics into a collection of temporally localized modeling tasks using three core components:

  1. Time-Conditioned Anchors: The scene geometry is represented by a sparse set of anchors initialized from periodically sampled keyframes. Each anchor is bound to a specific spatial position, scale, and discrete temporal index. Instead of moving across space, these anchors act as local priors that control the appearance and disappearance of local Gaussian primitives.
  2. Temporal Windowing Strategy: To keep temporal complexity under control during training and inference, the framework uses a sliding temporal window. Only anchors relevant to the queried time step are activated, preventing interference from distant frames and ensuring smooth temporal transitions.
  3. Hierarchical Spatio-Temporal Features: To stabilize Gaussian generation, each anchor aggregates three levels of features:
  4. Anchor features ($f_a$): Independent, learnable features for fine-grained local control.
  5. Static spatial features ($f_s$): Queried from a shared spatial grid to enforce local spatial consistency and prevent temporal jitter.
  6. Local temporal features ($f_t$): Queried from a partitioned set of 4D hash grids to capture fast-changing dynamics within localized time intervals.

A lightweight MLP decoder takes these fused features and generates the final parameters (position, rotation, scale, opacity, color) of the Temporal Gaussians used for volumetric rendering.

Overview of the ATGS pipeline, showing keyframe initialization, temporal windowing, and hierarchical feature decoding into Temporal Gaussians.

Datasets

The authors evaluated ATGS on several challenging multi-view, long-sequence benchmarks:

  • N3DV (Neural 3D Video Dataset): Includes scenes with up to 1200 frames captured from 19–21 cameras.
  • VRU (VRU-basketball): A highly challenging benchmark featuring large-scale, fast athletic motions across 1400 frames, captured by 36 cameras.
  • MeetRoom: A sparse-view dataset consisting of forward-facing scenes captured by 11–12 cameras.
  • SelfCap & PKU-DyMVHumans: Large-scale datasets containing complex human motions over thousands of frames used to validate generalization capability.

Key Insights and Ideas

  • Anchors over Trajectories: Explicitly tracking fast-moving points across thousands of frames is a recipe for optimization failure. Treating motion as the localized appearance/disappearance of Gaussians anchored in space-time simplifies the optimization landscape.
  • Feature Disentanglement Matters: Merely adding temporal features to an anchor can cause static and dynamic elements to blend, leading to a 10% training slowdown and blurry results. Explicitly separating the features into a shared static spatial grid and localized temporal hash grids allows the model to decouple static background geometry from rapid foreground motion.
Fig. 1. Based on long-sequence multi-view videos, our method is capable of reconstructing high-quality volumetric videos and enables novel view synthesis (NVS) at arbitrary viewpoints and time steps. Compared to prior approaches, which only support 20 frames fast motion or 300 frames minor hand or head motion (N3DV), our method supports fast motions over thousands.

Key Results

ATGS demonstrates significant quantitative and qualitative improvements over prior state-of-the-art methods like 4DGS and LocalDyGS:

  • Long-Sequence Capability: ATGS successfully reconstructs a 1400-frame sequence (about 1 minute of video) in a single training run, representing a 70× increase in temporal coverage over methods like LocalDyGS, which typically break down after 20 frames of fast motion.
  • Rendering Quality: On the VRU Long dataset, ATGS achieves a PSNR of 24.78 dB, outperforming LocalDyGS (23.21 dB) and 4DGS (23.01 dB). On the N3DV benchmark, it achieves a state-of-the-art PSNR of 32.56 dB.
  • Artifact Reduction: Qualitative results show that ATGS preserves fine-grained details (such as human hands and fast-moving basketballs) without the severe blurring or missing body parts common in streaming or deformation-based baselines.

Limitations and Future Work

Despite its strong performance, the authors note a few limitations:

  • Offline Paradigm: ATGS is an offline reconstruction method and does not currently support real-time training or streaming processing.
  • COLMAP Dependency: The framework relies on COLMAP for initial camera poses and sparse point clouds, which introduces a heavy computational bottleneck during preprocessing.
  • Sparse View Jitter: In regions with extremely sparse camera coverage (e.g., distant background audience areas), mild temporal jitter can still occur due to under-constrained optimization. Incorporating stronger temporal regularizers or geometric priors remains an open direction for future research.

Summary updated

Abstract

Volumetric video enables immersive free viewpoint rendering of dynamic real world scenes, yet existing methods struggle with long sequences and complex motions, often leading to temporal instability and visual artifacts. To address these challenges, we propose \ourname, a Gaussian splatting based framework for volumetric video reconstruction. Our key insight is that explicitly tracking long term complex motion with individual Gaussian primitives is inherently unstable. Instead, we organize Gaussians around time conditioned anchors that localize their spatial and temporal support, thereby reducing long range motion complexity. We further introduce a temporal windowing strategy to activate only anchors relevant to the queried time, which improves scalability and temporal coherence. In addition, to ensure spatial and temporal stability, we design a compact set of multi level anchor features that encode global features, local spatial features, and local temporal features, jointly constraining Gaussian generation. Extensive experiments demonstrate that \ourname \ consistently outperforms prior methods on long sequence volumetric videos with complex motions. Project page: https://github.com/WuJH2001/ATGS.
BibTeX
@article{wu2026atgs,
  author = {Jiahao Wu and Jie Liang and Die Hu and Jiayu Yang and Kaiqiang Xiong and Xiang Li and Xiaoyun Zheng and Chao Wang and Ronggang Wang},
  title = {{ATGS: Anchored Temporal Gaussian Splatting for Long Volumetric Video Representation}},
  journal = {ACM Trans. Graphics},
  year = {2026},
  doi = {10.1145/3811306},
}
Figure