TL;DR
KC-3DGS introduces a plug-and-play, wavelet-domain regularization strategy for 3D Gaussian Splatting (3DGS) to tackle the problem of oversmoothing and structural artifacts in sparse-view settings. By constraining higher-order natural image statistics—specifically kurtosis concentration and cross-band covariance—it forces the optimization to preserve fine textures and sharp details that standard pixel-space losses miss.
What Problem Does This Paper Address?
3D Gaussian Splatting (3DGS) has revolutionized real-time novel view synthesis, but it hits a wall when training data is sparse (e.g., only a dozen views of a scene). Under limited supervision, 3DGS optimization becomes highly underconstrained, frequently producing "floaters" (spurious floating artifacts), blurry blobs, and severe oversmoothing.
The root cause lies in standard photometric loss functions like $L_1$ and SSIM. These metrics operate in pixel space and only constrain aggregate reconstruction error. This creates a mathematical blind spot: the optimization algorithm can freely redistribute error across different frequency scales. As long as the average pixel difference is low, the model can smooth out high-frequency details (like textures and sharp edges) without being penalized.
Methods
To close this loophole, the authors propose KC-3DGS, which adds a lightweight, fully differentiable wavelet-domain regularization framework to standard 3DGS pipelines.
The method applies a 3-level discrete wavelet transform (DWT) using Daubechies-3 wavelets to both the rendered and ground-truth images, breaking them down into multi-scale frequency subbands.
It then applies three distinct losses:
- Scale-Weighted Wavelet Alignment Loss ($L_{wav}$): Directly aligns wavelet coefficients between rendered and ground-truth images. Crucially, it exponentially upweights finer scales (by a factor of $2^j$) to counteract the natural $1/f^\alpha$ energy decay of natural images, ensuring high-frequency details get a strong gradient signal.
- Supervised Kurtosis Concentration Loss ($L_{KC}^{sup}$): Natural images exhibit heavy-tailed (high-kurtosis) distributions in the wavelet domain that are remarkably consistent across subbands. This loss forces the rendered image's subband kurtosis values and overall spread to match the ground truth, preventing any single frequency band from dominating.
- Supervised Cross-Band Covariance Penalty ($L_{CBC}^{sup}$): This penalty minimizes the statistical dependency (off-diagonal covariance) between different subbands, encouraging frequency specialization while ensuring the spatial correlation matches the ground truth (e.g., preserving sharp corners where multiple orientations intersect).
Datasets
The authors evaluated their method across five diverse benchmarks:
- MipNeRF360: Indoor and outdoor scenes, tested in both full-view and highly constrained 12-view sparse settings.
- Tanks&Temples: Large-scale outdoor geometry with wide-baseline views.
- MVImgNet: Object-centric multi-view dataset.
- DeepBlending: Bounded indoor scenes with challenging lighting and reflections.
- WRIVA-ULTRRA: A highly challenging, real-world outdoor dataset characterized by irregular camera distributions, noisy coverage, varying resolutions, and transient objects (used as a stress test for sparse-view robustness).
Key Insights and Ideas
- The Pixel-Space Blind Spot is Real: The authors mathematically prove (via Parseval's identity) that pixel-space $L_2$ losses admit a $(3J-1)$-dimensional family of indistinguishable perturbations. Optimization can completely scramble how error is distributed across scales without changing the pixel loss value. Subband supervision closes this loophole.
- Unsupervised Kurtosis is Degenerate: Simply minimizing the spread of kurtosis across subbands without an anchor is a trap—the model can satisfy it perfectly by rendering a completely blurry, uniformly Gaussian image. Grounding the kurtosis directly to the ground-truth image's statistics is essential for proper regularization.
- Decorrelation as a Likelihood: The cross-band covariance penalty is proven to be equivalent to minimizing the KL divergence between the joint distribution of subbands and a factorized Gaussian distribution, providing a solid probabilistic foundation for subband independence.
Key Results
- Perceptual Leap: On the rugged WRIVA-ULTRRA dataset, KC-3DGS achieved a 9.48% relative improvement in DreamSim (a metric for human visual similarity) while simultaneously improving PSNR, SSIM, and LPIPS.
- Sparse-View Robustness: In extreme 12-view sparse settings, the method boosted PSNR by up to 0.5 dB on MipNeRF360 while significantly reducing floaters and color bleeding around fine structures like foliage.
- Plug-and-Play Integration: The regularizer requires zero architectural changes and integrates seamlessly into existing 3DGS frameworks like Nerfstudio's Splatfacto (yielding an 8.51% DreamSim boost) and FasterGS (yielding a 12.24% boost).
Limitations and Future Work
- Over-Regularization in Dense Views: KC-3DGS is highly optimized for sparse-view regimes. When abundant training views are available (dense multi-view coverage), the frequency constraints can occasionally over-regularize the optimization, leading to minor drops in standard metrics like PSNR on datasets like MipNeRF360.
- Computational Overhead: Incorporating the 2D discrete wavelet transform adds noticeable computational overhead during training. For speed-optimized frameworks like FasterGS, it resulted in a 2.7× increase in training time because the unoptimized DWT becomes a bottleneck relative to the highly optimized CUDA rasterizer. The authors note mitigating this overhead as a key direction for future work.
Summary updated