When optimizing a Neural Radiance Field (NeRF) or a 3D Gaussian Splatting (3DGS) model, we rely almost entirely on 2D photometric loss. We ask the model to ensure that, when its 3D representation is projected back onto a 2D plane, the rendered pixels match the training images.
However, volume rendering is fundamentally under-constrained. A single pixel's color can be mathematically explained in infinite ways: by a solid surface at the correct depth, by a highly dense point right in front of the camera lens, or by a cloud of semi-transparent particles scattered throughout empty space. Without explicit structural guidance, optimization algorithms frequently take shortcuts. To explain away complex view-dependent effects, sensor noise, or sparse viewpoints, models generate spurious density clusters. In the literature, these are known as "floaters," "ghostly artifacts," or near-field "fog."
To eliminate these artifacts, a prominent research direction has emerged: anchoring volumetric density to explicit or implicit 3D geometric priors. By constraining where density is allowed to exist, these techniques force optimization to resolve the scene's true underlying structure.
Carving Empty Space via Pre-Boundary Suppression
The most direct way to eliminate floaters is to enforce a simple physical truth: the space between a camera and the first visible surface must be completely empty. If we can estimate where a surface begins, we can strictly penalize any density that appears before it.
This intuition is formalized by Huiqiang Sun et al. in Dynamic View Synthesis from Small Camera Motion Videos. They introduce Distribution-based Depth Regularization (DDR), which penalizes non-zero volume density in the spatial regions preceding estimated object boundaries. By ensuring that the space between the camera and the primary surface remains clear, this constraint prevents the model from placing foggy artifacts in front of the geometry. Similarly, Yuqun Wu et al. demonstrate in MonoPatchNeRF: Improving Neural Radiance Fields with Patch-based Monocular Guidance that restricting volumetric density to a narrow interval around surfaces estimated via structure-from-motion or monocular depth prevents the formation of near-camera density clouds.
Standard Volume Rendering:
Camera [================== floater? === floater? === TRUE SURFACE ]
Result: Foggy reconstructions, broken geometry
With Pre-Boundary Suppression:
Camera [----------------- EMPTY SPACE -------------| TRUE SURFACE ]
Result: Sharp boundaries, clean empty space
This concept of "carving" empty space also applies to unobserved regions and silhouettes. For instance, Ethan Weber et al. show in Nerfbusters: Removing Ghostly Artifacts from Casually Captured NeRFs that penalizing density in regions of 3D space falling outside the viewing frustums of all training cameras prevents the formation of spurious background artifacts. When object silhouettes are available, Yufan Zhang et al. leverage them in Seeing A 3D World in A Grain of Sand by applying a visual hull-constrained depth loss. Using an S-shaped weighting function, their method strictly penalizes the placement of 3D primitives outside the volume defined by object silhouettes, effectively carving the object out of empty space.
Ultimately, density must be justified by data. As Kiyohiro Nakayama et al. argue in ProvNeRF: Modeling per Point Provenance in NeRFs as a Stochastic Field, requiring 3D density to be justified by the distribution of camera locations that actually observed it (provenance) suppresses floaters in sparse-view reconstructions by ensuring geometry is only placed where sufficient triangulation evidence exists.
Anchoring Primitives to Implicit Surfaces
While pre-boundary suppression clears out empty space, we also need a mechanism to ensure that the density representing the object itself forms a cohesive, watertight structure. This is achieved by coupling explicit rendering primitives (like 3D Gaussians) or volumetric density fields with implicit surface representations, such as Signed Distance Fields (SDFs) or Truncated Signed Distance Functions (TSDFs).
An SDF maps any 3D coordinate to its distance from the nearest surface, where the surface itself is implicitly defined as the zero-level set. By forcing density to conform to an SDF, we prevent primitives from scattering or clustering arbitrarily.
In GSDF: 3DGS Meets SDF for Improved Neural Rendering and Reconstruction, Bo Dai et al. regulate the density of rendering primitives using an underlying SDF. This implicit geometric constraint prevents the formation of floaters without sacrificing the ability to model complex, view-dependent visual effects. Similarly, Chaoda Song et al. develop this idea for high dynamic range scenes in Expo-GS: Exposure-Aware Signed Distance Function in Gaussian Splatting for High Dynamic Range. They use an exposure-aware SDF to guide the density control of 3D primitives—determining exactly where to initialize new Gaussians and where to prune existing ones—ensuring they align with actual object surfaces rather than floating in space due to misleading color gradients.
This synergy works in both directions: explicit primitives enable fast rendering, while implicit surfaces enforce geometric sanity.
- X. R. Lyu et al. demonstrate in 3DGSR: Implicit Surface Reconstruction with 3D Gaussian Splatting that integrating an SDF as a geometric prior into 3DGS enables the reconstruction of accurate, continuous surfaces, overcoming the fragmented, point-like limitations inherent to standard Gaussian splatting.
- For real-time tracking and mapping, Guanghao Li et al. show in Geometrically Constrained Gaussian Splatting SLAM that using a TSDF hash grid as a geometric prior effectively guides explicit Gaussian ellipsoids to align with the scene's true geometry, significantly improving map reconstruction and camera pose estimation.
Regularizing Generative and Parametric Models
The problem of unconstrained density becomes even more severe in generative and monocular dynamic tasks, where the model must synthesize 3D structures from highly limited or single-view data. In these scenarios, researchers use known parametric templates (such as human body or face models) as the geometric anchor.
In AvatarGen: A 3D Generative Model for Animatable Human Avatars, Jianfeng Zhang et al. transition from a pure density field to an SDF representation. This shift allows them to apply structural priors that constrain the generated surface to remain close to a known parametric human template, eliminating surface noise and floating points. For facial synthesis, Hongyi Xu et al. introduce a geometry prior loss in OmniAvatar: Geometry-Guided Controllable 3D Head Synthesis that forces the synthesized neural density to conform to a 3D head model's SDF, ensuring strict anatomical accuracy.
Taking a direct regularization approach, Shangzhe Wu demonstrates in CGOF++: Controllable 3D Face Synthesis With Conditional Generative Occupancy Fields that a density regularization term penalizing neural density in regions of space distant from a reference 3D surface forces the generative model to compress its volumetric representation into a specific, clean geometric shape.
Unconstrained Generative Density:
[ Parametric Template ] --> Optimization places random density clouds
outside the template to match 2D textures.
Surface-Constrained Density:
[ Parametric Template ] --> Density outside the template is heavily penalized.
Representation is compressed strictly into the shape.
When animating these models over time, maintaining surface consistency is vital. Xuesong Li et al. address this in DGNS: Deformable Gaussian Splatting and Dynamic Neural Surface for Monocular Dynamic 3D Reconstruction by pruning visual primitives based on their distance from a learned dynamic zero-level set. This cross-representation supervision eliminates fragmented geometry and ensures that visual elements remain bound to a cohesive, moving surface.
Why It Works: Reducing the Solution Space
Geometric density constraints succeed because they fundamentally alter the optimization landscape. Photometric consistency alone creates a "many-to-one" mapping problem where a flawless 2D image can be rendered from a deeply flawed 3D scene full of floaters.
By introducing geometric constraints—whether by carving out pre-boundary space, penalizing deviations from an SDF zero-level set, or enforcing silhouette boundaries—we mathematically eliminate the degenerate, artifact-ridden solutions. The optimization algorithm can no longer cheat by placing a tiny, high-opacity cloud right in front of the lens or hiding floaters in unobserved zones; it is forced to distribute density precisely along the physical boundaries of the scene.
This structural grounding does more than just clean up visual artifacts; it ensures that the underlying 3D representation is a faithful reflection of reality. Consequently, these geometrically constrained models provide highly accurate depth maps and consistent spatial representations that can be reliably leveraged for downstream tasks, such as wide-baseline pixel alignment in multi-view image compression (Yujun Huang et al. in 3D-LMVIC: Learning-based Multi-View Image Compression with 3D Gaussian Geometric Priors) and robust image alignment across large camera disparities (Luca Savant Aira et al. in Deep 3D World Models for Multi-Image Super-Resolution Beyond Optical Flow).
Key Foundational Work
AvatarGen: A 3D Generative Model for Animatable Human Avatars (2022) — contributed 2 concept elements to this direction.
Nerfbusters: Removing Ghostly Artifacts from Casually Captured NeRFs (2023) — contributed 2 concept elements to this direction.
OmniAvatar: Geometry-Guided Controllable 3D Head Synthesis (2023) — contributed 2 concept elements to this direction.