Visibility Driven Pruning for Volumetric Representations

214 concepts126 papers1 proposal

Modern 3D reconstruction techniques like Neural Radiance Fields (NeRFs) and 3D Gaussian Splatting represent scenes using millions of explicit primitives—whether they are voxels, points, or anisotropic Gaussians. To make these representations compact, an intuitive first step is to prune primitives with low density or opacity. However, as Haotian Bai et al. point out in Dynamic PlenOctree for Adaptive Sampling Refinement in Explicit NeRF, density-based pruning suffers from a fundamental flaw: it retains primitives that are highly dense but permanently occluded from all viewing angles. These hidden elements consume substantial memory and slow down rendering pipelines without ever contributing a single pixel to the final output.

To solve this, a powerful paradigm has emerged: visibility-driven pruning. Instead of asking how dense a primitive is, these methods ask how much it actually contributes to what the camera sees. By evaluating occlusion and transmittance during or after training, we can safely discard the "invisible" interior of 3D models.

Tracking Ray Contribution and Transmittance

To determine if a 3D primitive is visible, we must track how light interacts with the scene volume during alpha-blending.

When a ray travels through a volumetric field, its energy is progressively attenuated by the density of the primitives it encounters. This is captured by accumulated transmittance—the probability that a ray has traveled up to a certain point without hitting anything. Lingzhi Li et al. leverage this in Compressing Volumetric Radiance Fields to 1 MB by calculating importance scores based on accumulated transmittance and ray termination probabilities. If a voxel sits behind a fully opaque surface, the transmittance drops to zero, meaning any subsequent voxels have zero probability of terminating the ray and are marked for pruning.

Similarly, Sharath Girish et al. introduce a transmittance-weighted influence metric in EAGLES: Efficient Accelerated 3D Gaussians with Lightweight EncodingS. This metric accounts for both an individual primitive's opacity and the cumulative occlusion from all preceding elements along the line of sight.

In the context of point-based rendering, Didier Stricker et al. demonstrate in Object-Centric 2D Gaussian Splatting: Background Removal and Occlusion-Aware Pruning for Compact Object Models that tracking when a pixel reaches full opacity allows the system to immediately identify and remove hidden internal structures. By shifting the pruning metric from static opacity to dynamic ray intersection frequency, as shown by Changbai Li et al. in Hierarchical Gaussian Splatting for Memory-Efficient High-Resolution 3D Reconstruction, models can eliminate up to half of their primitives with zero loss in visual fidelity.

Ray Direction ──>  [Visible Primitive]  ──>  [Occluded Primitive]
                   (Transmittance > 0)       (Transmittance → 0)
                   *RETAINED*                *PRUNED*

Mesh-Guided and Proxy-Based Occlusion Culling

While tracking ray statistics during training is highly effective, evaluating visibility for millions of primitives across arbitrary viewpoints can become computationally expensive. To accelerate this process, researchers have turned to hybrid representations that combine explicit geometry with volumetric fields.

Antoine Guedon et al. propose a solution in Gaussian Frosting: Editable Complex Radiance Fields with Real-Time Rendering by wrapping a traditional triangle mesh in a layer of 3D Gaussians. Because the underlying mesh provides an explicit surface boundary, it can be used to perform instant occlusion culling: any volumetric primitive residing behind the mesh from the camera's perspective is skipped during rendering.

To avoid the complexity of creating a perfect mesh, Yuning Gong et al. utilize a coarse, low-resolution geometric proxy in Proxy-GS: Efficient 3D Gaussian Splatting via Proxy Mesh. By leveraging dedicated hardware rasterization units to generate real-time depth maps of this proxy, the system can perform rapid visibility testing. Comparing primitive coordinates against these hardware-generated depth buffers allows the renderer to filter out occluded components in less than a millisecond, tripling the rendering frame rate.

This geometric guidance is also valuable during optimization; for instance, Dor Verbin demonstrates in Nuvo: Neural UV Mapping for Unruly 3D Representations that prioritizing visible surface points while ignoring internal geometry prevents the fragmentation of texture atlases.

Continuous Selection and Opacity Decay

Rather than applying hard thresholds post-training, some of the most robust visibility frameworks integrate selection mechanisms directly into the optimization loop.

One approach is to apply a global continuous decay to the opacity of all spatial primitives during training. As shown by Liang Han et al. in Binocular-Guided 3D Gaussian Splatting with View Consistency for Sparse View Synthesis, when a global decay coefficient is applied, only the primitives that actively receive strong optimization gradients from the scene's visible surfaces can counteract the decay. Redundant or floating elements that do not contribute to view consistency naturally fade away and are pruned.

For test-time efficiency, Zhigang Cheng et al. introduce a distance-based primitive masking strategy in CLoD-GS: Continuous Level-of-Detail via 3D Gaussian Splatting. By applying a learnable decay factor that attenuates opacity based on viewing distance, primitives whose contribution falls below a specific threshold are dynamically filtered out via a boolean mask. This bypasses the need for complex hierarchical data structures while maintaining high frame rates for distant views.

Why It Works: The Benefits of Going Hollow

Visibility-driven optimization works because it exploits a fundamental truth of 3D scenes: we only ever see the outer shell. By making solid volumetric objects "hollow," we eliminate the massive memory overhead of internal structures.

Beyond simple storage savings, removing non-visible features has profound structural benefits for neural representations. For example, in hashgrid-based architectures like Instant-NGP, multiple 3D coordinates map to the same slots in a shared multiresolution table, causing hash collisions that degrade image quality.

Xiufeng Xie et al. demonstrate in HollowNeRF: Pruning Hashgrid-Based NeRFs with Trainable Collision Mitigation that pruning the features of invisible or internal spatial points drastically reduces these collisions. By clearing out the signal from non-visible regions, the model prevents internal interference, allowing the remaining parameters to focus entirely on representing visible surfaces with high fidelity.

Ultimately, focusing computation strictly on what is visible allows frameworks like the one proposed by Lingzhi Li et al. in Compact Real-time Radiance Fields with Neural Codebook to achieve massive reductions in storage while maintaining real-time rendering speeds.

Key Foundational Work

Portals and mirrors (1995) — contributed 6 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_visibility_based_rendering_optimization_202621fb,
  title = {visibility-based rendering optimization},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/visibility-based-rendering-optimization/rdr_202621fba19547ff8562df54888e577d},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-21},
}

This exact version (version 1, frozen text):

@misc{lacuna_visibility_based_rendering_optimization_202621fb_v1,
  title = {visibility-based rendering optimization},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/visibility-based-rendering-optimization/rdr_202621fba19547ff8562df54888e577d/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-21},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake