Abstract: 3D Gaussian Splatting has emerged as a powerful approach in novel view synthesis, delivering rapid training and rendering but at the cost of an ever-growing set of Gaussian primitives that strains memory and bandwidth. We introduce AutoOpti3DGS, a training-time framework that automatically restrains Gaussian proliferation without sacrificing visual fidelity. The key idea is to feed the input images to a sequence of learnable Forward and Inverse Discrete Wavelet Transforms, where low-pass filters are kept fixed, high-pass filters are learnable and initialized to zero, and an auxiliary orthogonality loss gradually activates fine frequencies. This wavelet-driven, coarse-to-fine process delays the formation of redundant fine Gaussians, allowing 3DGS to capture global structure first and refine detail only when necessary. Through extensive experiments, AutoOpti3DGS requires just a single filter learning-rate hyper-parameter, integrates seamlessly with existing efficient 3DGS frameworks, and consistently produces sparser scene representations more compatible with memory or storage-constrained hardware.
Introduction
3D Gaussian Splatting (3DGS) has revolutionized novel view synthesis by offering real-time rendering and fast training. However, it has a major memory and storage footprint problem. As the model trains, its Adaptive Density Control (ADC) mechanism rapidly spawns new Gaussian primitives to capture fine scene details, often resulting in millions of redundant splats that strain hardware resources.
Previous attempts to mitigate this proliferation often rely on complex pruning pipelines or rigid, hand-crafted blurring schedules (like Opti3DGS) that fail to adapt to different datasets. AutoOpti3DGS addresses this by introducing a fully automatic, dataset-adaptive coarse-to-fine training curriculum.
Methodology: Learnable Wavelet Transforms
The core innovation of AutoOpti3DGS is replacing manual image blurring with a sequence of Learnable Forward and Inverse Discrete Wavelet Transforms (DWT) applied directly to the input images during training.
This mechanism works through a clever "zero initialization" trick:
- Wavelet Decomposition: The Forward DWT splits an input image into frequency subbands: one low-pass (coarse structure) and three high-pass (fine details).
- The "Zero" Trick: While low-pass filters are fixed, high-pass filters are learnable and initialized to zero. Initially, the reconstructed image lacks all fine details, forcing the 3DGS model to lay down a sparse foundation of large Gaussians to capture global shapes.
- Automatic Sharpening: An auxiliary "wavelet loss" penalizes the difference between learnable high-pass filters and fixed orthogonal Haar wavelet references. As training progresses, the filters converge, gradually introducing high-frequency details into the training images.
Key Results and Insights
By manipulating what the model sees rather than hacking the internal density control, AutoOpti3DGS effectively controls primitive proliferation.
- Efficiency Gains: On the LLFF dataset, the framework achieved a 20.39 PSNR using only 224K Gaussians, compared to Vanilla 3DGS's 20.40 PSNR with 272K Gaussians—an ~18% reduction in primitives with virtually zero loss in quality.
- Compounding Compression: Because it modifies input images, it is agnostic to the underlying architecture. When paired with Mini-Splatting, it pushed the Gaussian count down to an incredibly sparse 38K (from 49K) while slightly improving PSNR.
Conclusion
AutoOpti3DGS provides an elegant, mathematically grounded curriculum for 3DGS training. By using learnable wavelets to manage the transition from coarse to fine details, it ensures that high-frequency primitives are only generated when essential, resulting in significantly more efficient 3D representations suitable for memory-constrained environments.
Summary updated