Resolving Sparsity Artifacts in Point Based Rendering

24 concepts24 papers

When we capture the 3D world using LiDAR, depth cameras, or multi-view stereo, we naturally record data as a collection of discrete samples: a point cloud. While computationally efficient and flexible, representing a continuous world with zero-dimensional points introduces a fundamental geometric flaw known as the surface continuity gap.

If you look at a point cloud from the exact perspective it was captured, it appears solid. But as soon as you move the camera, change the scale, or warp the viewpoint, the illusion shatters. The underlying surface breaks down into a sparse collection of disconnected coordinates. Understanding why these artifacts happen—and how modern graphics pipelines patch them to build continuous surfaces—is central to modern 3D vision.

The Anatomy of a Hole

Why do sparse point clouds fail when rendered? The issue stems from how discrete elements behave under perspective transformations.

Captured View:

*  *  *  *  *  (Points appear dense and continuous)
                 \  |  |  |  /
                  \ |  |  | /
Virtual Camera:     \      /
                     v    v
Novel View:

*   *   *   *   *  (Points spread apart, creating holes)

When a novel view is synthesized, image features or 3D points are projected onto a new image plane via forward-warping or splatting. As shown by Jia Zheng et al. in Layout-Guided Novel View Synthesis From a Single Indoor Panorama, this process stretches discrete pixels across larger target areas. Because the representation lacks inherent connectivity, this stretching leaves empty gaps, creating a sparse distribution characterized by numerous small holes.

This degradation worsens under specific conditions:

To overcome this, researchers have developed three core strategies: continuous splatting kernels, local neural fields, and alternative continuous geometries.

Strategy 1: Continuous Splatting Kernels

The most direct way to close the surface continuity gap is to replace zero-dimensional points with continuous, differentiable spatial distributions. Instead of projecting a single pixel coordinate, the pipeline projects a smooth mathematical function that can expand or contract to fill empty space.

Yueyu Hu et al. leverage this in Low Latency Point Cloud Rendering with Learned Splatting by predicting 3D elliptical Gaussian parameters for every point in a scene. Because Gaussians have continuous spatial extent, they naturally overlap and blend with their neighbors, producing hole-free, smooth surfaces during rasterization.

Discrete Points:      .         .         .         .

Gaussian Kernels:   (***)     (***)     (***)     (***)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                     Continuous, overlapping coverage

When dealing with highly sparse or non-uniform data, static kernels are not enough. To prevent semi-transparent artifacts and noisy geometry, Changfeng Ma et al. introduce a dynamic approach in Sparse Point Cloud Patches Rendering via Splitting 2D Gaussians, where 2D Gaussians are adaptively predicted and "split" to fill geometric gaps on the fly.

By coupling point-wise Gaussian regression with neural feature splatting, frameworks like the one proposed by Jiaxu Wang et al. (PFGS: High Fidelity Point Cloud Rendering via Feature Splatting) ensure full surface coverage during 2D projection, eliminating the surface bleeding and aliasing common to traditional point graphics. This spatial smoothness constraint is critical; as demonstrated by Luca Savant Aira et al. in Gaussian Splatting for Efficient Satellite Image Photogrammetry, without these continuous constraints, models trained on sparse data inevitably collapse into noisy, disjointed representations.

Strategy 2: Local Neural Fields and Implicit Manifolds

Rather than smoothing the points during rendering, another approach anchors continuous mathematical functions directly to the points in 3D space. This treats the point cloud not as the surface itself, but as a "scaffold" supporting a continuous field.

In Neural Points: Point Cloud Representation with Neural Fields for Arbitrary Upsampling, Wanquan Feng et al. redefine the point cloud paradigm by treating each point as the center of a local neural field. When the surface is magnified, the pipeline queries these local fields rather than interpolating between raw coordinates. This allows for arbitrary-factored upsampling while maintaining a perfectly smooth underlying manifold.

Point Scaffold:       o                 o                 o
                       \               / \               /
Local Fields:        [--neural field--]   [--neural field--]
                     =======================================
                     Continuous Implicit Surface Manifold

Similarly, Shi-Lin Liu et al. combine the computational efficiency of points with the topological consistency of implicit functions in Deep Implicit Moving Least-Squares Functions for 3D Reconstruction. They use a deep network to generate a scaffold of oriented points that collectively define a global, continuous surface via Implicit Moving Least-Squares (IMLS).

When point density is insufficient, neural feature interpolation can still break down and cause localized spherical distortions on the surface (Shih-Yang Su et al. in NPC: Neural Point Characters from Video). To resolve this, Yiping Xie et al. develop DiffPBR: Point-Based Rendering via Spatial-Aware Residual Diffusion. Instead of relying purely on geometric interpolation, they splat 3D-consistent noise directly from the point cloud and use a spatial-aware residual diffusion process to seamlessly patch holes, ensuring photorealistic and view-consistent surface generation.

Strategy 3: Alternative Continuous Geometries

A third school of thought argues that to truly solve discretization artifacts, we must abandon point clouds and voxel grids entirely in favor of primitives with explicit or implicit continuity.

Historically, volumetric methods like binary occupancy grids or voxels have been used to represent 3D structures, but they suffer from severe cubic scaling laws and produce blocky, jagged approximations (Edmond Boyer et al. in On Computing Exact Visual Hulls of Solids Bounded by Smooth Surfaces). As explained by the authors of Octree Generating Networks: Efficient Convolutional Architectures for High-resolution 3D Outputs, binary voxel representations only identify the presence or absence of geometry, failing to capture rich, continuous surface details. This has driven a shift toward Signed Distance Functions (SDFs), as seen in Locally Attentional SDF Diffusion for Controllable 3D Shape Generation, which map continuous mathematical boundaries to capture smooth, high-resolution geometry.

For explicit rendering, researchers have introduced continuous geometric primitives that bridge the gap between flexible points and rigid meshes:

By moving from discrete coordinates to continuous representations—whether through overlapping Gaussian splats, localized neural fields, or deformable geometric sheets—modern 3D vision pipelines effectively bridge the surface continuity gap, turning sparse sensor data into watertight, photorealistic digital worlds.

Key Foundational Work

Efficient Non-Parametric Surface Representations Using Active Sampling for Push Broom Laser Data (2010) — contributed 1 concept elements to this direction.
On Computing Exact Visual Hulls of Solids Bounded by Smooth Surfaces (2001) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_point_cloud_sparsity_artifacts_eec05ffb,
  title = {point cloud sparsity artifacts},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/point-cloud-sparsity-artifacts/rdr_eec05ffb96284410a5f43f00b981ec77},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_point_cloud_sparsity_artifacts_eec05ffb_v1,
  title = {point cloud sparsity artifacts},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/point-cloud-sparsity-artifacts/rdr_eec05ffb96284410a5f43f00b981ec77/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}

See something wrong? Report a mistake