DimFlow: Bidirectional Grid-Expert Learning for Dynamic Gaussian Reconstruction

Gabriel Adineera, Jinoh KimarXiv 2026 · arXiv

TL;DR

DimFlow introduces a bidirectional learning framework that significantly boosts the quality of dynamic 3D Gaussian reconstruction. By pairing a fast multi-resolution grid with a coordinate-based "expert" MLP and using adversarial alignment to sync them, it achieves a nearly 2 dB PSNR improvement over the state-of-the-art 4D-GS while maintaining real-time rendering speeds (85 FPS).

What Problem Does This Paper Address?

Reconstructing moving scenes in 3D is a balancing act between speed, storage, and visual fidelity. Recent methods like 3D Gaussian Splatting (3DGS) have revolutionized static scenes, but dynamic extensions often struggle.

Current approaches typically fall into two camps:

  1. Grid-based methods: These are fast and store local details efficiently but can lack global consistency across space and time.
  2. Coordinate-based MLPs: These capture global regularities well but are computationally expensive to query.

The authors identify a key gap: simply sticking these two components together in a pipeline doesn't mean they will work together effectively. The challenge is coordinating spatial detail, motion, and appearance in a way that remains efficient to query while ensuring features are compatible across the entire scene's timeline.

Methods

DimFlow tackles this through Bidirectional Grid–Expert Learning. The architecture consists of two main feature branches that interact during both the forward pass and training:

  • Multi-resolution Grid Branch: Provides interpolated local features at various scales using factorized coordinate planes (e.g., $xy, xt, zt$).
  • Coordinate Expert Branch: A shared MLP that evaluates spatial and temporal encodings to provide a global perspective.
  • Adversarial Alignment: To ensure these two different representations "speak the same language," the authors use a discriminator during training. This forces the grid's feature distribution to match the expert's distribution.
  • Adaptive Fusion: A gating mechanism dynamically blends features from both branches based on scene complexity and motion descriptors (velocity and acceleration).
  • Specialized Attribute Experts: The fused features are fed into four dedicated branches that predict updates for Gaussian deformation, rotation, opacity, and temporal information.
The DimFlow pipeline: features from the grid and MLP are fused, gated, and then decoded into Gaussian updates.

Datasets

The model was evaluated on a mix of synthetic and real-world dynamic sequences:

  • D-NeRF: Synthetic motion sequences.
  • HyperNeRF: Real-world sequences with topological variations.
  • DAVIS: A standard benchmark for video object segmentation and dynamic scene analysis.

Key Insights and Ideas

  • Distribution vs. Correspondence: The authors point out that matching feature distributions (via adversarial loss) is not enough—you also need pointwise consistency. Figure 1 illustrates how adversarial alignment creates a much stronger overlap between the grid and expert feature spaces, which directly translates to better reconstruction.
  • The "Expert" as a Teacher: The coordinate MLP acts as a global expert that guides the local grid features. The "bidirectional" nature refers to the forward feature enhancement and the backward optimization feedback (residual signals) sent to the grid.
  • Compensating for Temporal Lag: Standard temporal smoothing often introduces lag. DimFlow uses a finite-difference update with a trend compensation term ($\eta$) to balance regularity with responsiveness to rapid motion.
t-SNE projections showing how adversarial alignment forces the grid and expert features into a shared distribution.

Key Results

DimFlow sets a new high bar for dynamic reconstruction quality without the typical "quality tax" on rendering speed:

  • Accuracy: Achieved 39.73 dB PSNR in aggregate, a 1.92 dB improvement over 4D-GS and 1.41 dB over DG-Mesh.
  • Speed: Maintained a rendering rate of 85 FPS, matching the speed of 4D-GS.
  • Efficiency: The model size is a compact 18.9 MB, only a marginal (3.3%) increase over 4D-GS.
  • Ablation Impact: Removing the coordinate expert framework caused the largest quality drop (2.03 dB), proving that the interaction between branches is the "secret sauce" of the system.
Comparison of PSNR vs. Rendering Speed. DimFlow sits at the top-right, offering the best quality at high speeds.

Limitations and Future Work

While DimFlow performs exceptionally well, the authors acknowledge several hurdles:

  • Input Dependency: The system relies heavily on accurate camera poses and the information present in the input views.
  • Complex Physics: It still struggles with "fluid-like" motion, extreme topology changes, and areas hidden by occlusion.
  • Physical Consistency: The model learns a representation that looks right, but it doesn't guarantee that the underlying Gaussian trajectories are physically plausible.
  • Future Directions: The authors suggest integrating richer motion priors and exploring "budgeted execution," where the gate could decide to skip expensive expert queries in simple scene regions to save power or time.

Summary updated

Abstract

Dynamic Gaussian reconstruction requires compact scene features to support coordinated changes in geometry and appearance. A factorized grid provides efficient local queries, but accurate interpolation alone does not ensure that the resulting dynamic attributes remain compatible across space and time. We introduce DimFlow, a framework that learns these features through bidirectional interaction between a multi-resolution grid and a coordinate-based expert branch. Feature-level adversarial alignment connects their distributions, adaptive fusion transfers complementary information, and activation-masked residual feedback refines the grid during training. Four attribute experts organize deformation, rotation, opacity, and temporal information before Gaussian rendering. In the aggregate comparison across D-NeRF, HyperNeRF, and DAVIS, DimFlow achieves 39.73 dB PSNR, improving on 4D-GS by 1.92 dB while matching its reported 85 FPS rendering speed. The model occupies 18.9 MB, compared with 18.3 MB for 4D-GS. Component and attribute-expert ablations reveal complementary contributions from feature alignment, adaptive exchange, and temporal processing.
BibTeX
@misc{adineera2026dimflow,
  author = {Gabriel Adineera and Jinoh Kim},
  title = {{DimFlow: Bidirectional Grid-Expert Learning for Dynamic Gaussian Reconstruction}},
  year = {2026},
  eprint = {2610.34952},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2610.34952},
}
Figure