TL;DR
ProGS is a new codec for 3D Gaussian Splatting (3DGS) that organizes scene data into a hierarchical octree, enabling "progressive coding"—where a coarse version of a scene loads instantly and sharpens as more data arrives. It achieves a massive 45× reduction in storage compared to original 3DGS while actually improving visual quality by over 10%.
What Problem Does This Paper Address?
3D Gaussian Splatting (3DGS) has taken the computer graphics world by storm because it allows for incredibly high-quality, real-time rendering of 3D scenes. Unlike Neural Radiance Fields (NeRFs), which require expensive ray-marching through a neural network, 3DGS uses millions of "learnable" 3D ellipsoids (Gaussians) that are "splatted" onto the screen.
The catch? These millions of Gaussians create massive files, often hundreds of megabytes or even gigabytes for a single scene. While researchers have found ways to compress this data, most existing methods produce "static" files: you have to download the entire thing before you can see anything. This is a dealbreaker for streaming applications (like VR or web-based 3D maps) where network bandwidth fluctuates. We need progressive coding—the 3D equivalent of how a low-res JPEG appears first on a slow connection and gradually becomes crisp.
Methods
The authors introduce ProGS, a streaming-friendly codec built on three main pillars:
- Octree-Based Anchors: Instead of storing millions of individual Gaussians, ProGS uses "anchors" (inspired by Scaffold-GS) organized into an octree. Each level of the octree represents a different Level-of-Detail (LoD). Level 1 provides a blurry, low-res "scaffold" of the scene, while Level 5 adds the fine-grained textures.
- Mutual Information (MI) Enhancement: A major challenge with hierarchical structures is that the lower levels often look terrible because they lack detail. ProGS uses a specialized InfoNCE loss and a coarse-to-fine optimization strategy. This forces parent nodes in the octree to "inherit" more informative content from their children, ensuring that even the earliest stages of a stream look usable.
- Adaptive Anchor Adjustment: During training, the system dynamically "grows" or "prunes" branches of the octree based on where the scene needs more detail. This ensures the bit budget is spent on complex areas (like a person's face) rather than empty space.
- Context-Based Entropy Coding: To squeeze the data further, ProGS uses a hash-grid and lightweight MLPs to predict the probability distribution of anchor attributes. This allows an arithmetic coder to pack the data into a highly compact binary bitstream.
Datasets
The researchers evaluated ProGS on three standard benchmarks for novel view synthesis:
- Mip-NeRF360: A dataset of complex 360-degree indoor and outdoor scenes.
- BungeeNeRF: Focuses on multi-scale scenes (from city-scale to close-ups).
- Tanks and Temples: High-resolution scans of large-scale outdoor environments.
Key Insights and Ideas
- The "Alignment" Hypothesis: The authors hypothesized that the representation goals of low and high LoDs are fundamentally aligned. By maximizing the Mutual Information between parent and child nodes, they proved that lower levels can be trained to "anticipate" the details of higher levels, leading to much smoother visual transitions during streaming.
- Structure over Sparsity: While previous methods focused on simply deleting "useless" Gaussians, ProGS shows that organizing them into a strict, voxel-aligned octree is actually more efficient for compression because it allows for deterministic position reconstruction (saving 23 bits per anchor).
- Streaming-First Design: The bitstream is "front-loaded" with a small header (hash-grid and MLP weights), followed by independent chunks for each LoD. This allows a decoder to start rendering immediately after the first chunk arrives.
Key Results
- Massive Compression: ProGS achieves up to a 45× reduction in file size compared to the original 3DGS format.
- Better Quality: Despite the compression, it improves PSNR (a measure of visual fidelity) by over 10% compared to baseline 3DGS.
- Progressive Performance: At the coarsest level (LoD 1), the file size is reduced by over 144×, yet it still provides a recognizable, usable representation of the scene.
- Real-Time Speeds: The system maintains high framerates (often 150+ FPS), making it viable for real-time VR/AR applications.
Limitations and Future Work
The primary trade-off is training time. Because ProGS has to manage the octree structure and multi-level rendering during optimization, it takes roughly 3× longer to train than Scaffold-GS.
The authors suggest that future work could explore real-time adaptive encoding, where the server adjusts the octree structure on the fly based on the user's specific viewpoint and available bandwidth, rather than relying on a pre-optimized hierarchy.
Summary updated