Neural Surface Continuity for Sparse Point Clouds

This direction is no longer present in the current build. Its last version stays readable and citable here.

Version 1 of 144 papers47 conceptsfrozen 2026-10-04

The fundamental challenge in 3D computer vision is that the world is continuous, but our sensors are discrete. Whether captured via LiDAR, depth cameras, or multi-view stereo, raw 3D data usually arrives as a point cloud: a collection of isolated coordinates in space.

While point clouds are precise, they are "holey." When you try to render them or reason about their geometry, you encounter the "sparsity gap." If you look at a point cloud from a new angle, the space between the points opens up, leading to visual artifacts like tearing, background bleeding, and a lack of solid surfaces. Recent research has shifted away from treating points as discrete entities, instead developing mechanisms to "fill the gaps" by enforcing neural surface continuity.

The Problem of Hard Projections

Traditionally, to turn a 3D point cloud into a 2D image, researchers used "hard" projections. You take a point, find its corresponding pixel on a camera plane, and paint it. However, as Jia Zheng and colleagues note in Layout-Guided Novel View Synthesis From a Single Indoor Panorama, this forward-warping creates a sparse distribution of data. Because pixels are stretched across larger areas in the target view, the result is a "salt-and-pepper" image full of empty holes.

This isn't just a rendering problem; it's a learning problem. In SplAttN: Bridging 2D and 3D with Gaussian Soft Splatting and Attention for Point Cloud Completion, Zhaoyang Li describes this as Cross-Modal Entropy Collapse. When a model tries to map 3D geometry to 2D features, discrete mappings leave regions with zero gradients. If there is no data in a pixel, the neural network cannot "see" the underlying geometry to improve it.

Aggregating Features in 3D Volumes

One way to solve this is to stop projecting onto a flat 2D plane and start projecting into a 3D volume. Peng Dai and Yinda Zhang pioneered this in Neural Point Cloud Rendering via Multi-Plane Projection. Instead of a single image, they project point features into a "frustum volume"—a stack of planes at different depths.

By using a volume, the network can learn to aggregate features across space. If a point is missing in one spot, the network can interpolate from its neighbors in 3D. Tao Hu refined this intuition in TriVol: Point Cloud Rendering via Triple Volumes, decomposing the scene into three axis-aligned feature volumes. This "triple slim volume" acts as a continuous bridge, allowing the model to synthesize photo-realistic surfaces even when the input points are incredibly sparse.

Softening the Point with Gaussian Splatting

A more recent and highly effective intuition is to treat points not as infinitely small dots, but as "fuzzy" kernels. This is the core of Gaussian Splatting.

In Sparse Point Cloud Patches Rendering via Splitting 2D Gaussians, Changfeng Ma and Jie Guo demonstrate that by "splitting" 2D Gaussians, you can cover the gaps between sparse points. Similarly, Chen Mu uses this in TGP to solve occupancy prediction. While sparse points provide precise localization, Gaussians provide the "volumetric fullness" needed to represent solid objects like cars or trees. By "softening" the points into Gaussians, the representation becomes differentiable and continuous, allowing the model to fill in unobserved regions through standard optimization.

Anchoring Implicit Fields

Perhaps the most robust way to achieve continuity is to use the sparse points as "anchors" for a continuous neural field, such as a Neural Radiance Field (NeRF).

In Point2Pix: Photo-Realistic Point Cloud Rendering via Neural Radiance Fields, Tao Hu uses point clouds to guide ray sampling. Instead of searching all of empty space, the model only looks near the points. However, because it uses a NeRF backbone, it inherently produces a continuous signal between those points.

Jonas Kulhanek takes this further in Tetra-NeRF: Representing Neural Radiance Fields Using Tetrahedra. By performing a Delaunay triangulation on the point cloud, they create a tetrahedral mesh. This mesh defines a continuous field where features are interpolated using barycentric coordinates. This ensures that every ray passing through the scene hits a "surface" of features, effectively banishing the "hole" problem entirely.

Why It Works: The Smoothness Prior

The recurring theme across these papers—from Co-SLAM to REDRESS—is the introduction of a smoothness prior.

Discrete points have no inherent relationship with their neighbors. By moving to volumes, Gaussians, or tetrahedral meshes, researchers are forcing the neural network to assume that the world is made of continuous surfaces. As Dominik Zimny shows in Points2NeRF, once you have a continuous representation, you can even interpolate in a latent space to generate entirely new, watertight 3D objects from just a handful of scattered points. The "magic" isn't in the points themselves, but in the neural architecture's ability to imagine the surface that connects them.

Papers in this version

History & citationVersion 1 · 2026-10-04

No longer present in the current build · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_neural_surface_continuity_for_sparse_poi_a1c46235,
  title = {Neural Surface Continuity for Sparse Point Clouds},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/neural-surface-continuity-for-sparse-point-clouds/rdr_a1c46235b64e48e3b28945dcb0622210},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-22},
}

This exact version (version 1, frozen text):

@misc{lacuna_neural_surface_continuity_for_sparse_poi_a1c46235_v1,
  title = {Neural Surface Continuity for Sparse Point Clouds},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/neural-surface-continuity-for-sparse-point-clouds/rdr_a1c46235b64e48e3b28945dcb0622210/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-22},
}