Reconstructing Complete 3D Geometry from Sparse LiDAR Scans

You are reading version 1 of this direction, frozen 2026-10-04. The latest version is version 2.

Version 1 of 21,183 papers1,413 conceptsfrozen 2026-10-04

LiDAR sensors are the primary "eyes" of autonomous vehicles, yet they provide a fundamentally incomplete picture of the world. Because LiDAR operates by firing discrete laser beams that reflect off the first surface they hit, the resulting point clouds are inherently "holey." A car doesn't see a 3D volume; it sees a sparse, one-sided shell of the side of an object facing the sensor.

The research direction of 3D Point Cloud and Semantic Scene Completion focuses on "hallucinating" the missing geometry. By leveraging learned shape priors, temporal context, and generative models, researchers are moving from simply detecting points to reconstructing full, dense 3D environments.

The Challenge of Partial Visibility

In a single LiDAR sweep, objects are subject to two types of data loss. First, sparsity occurs because laser beams diverge with distance, leaving large gaps between points on far-away objects. Second, occlusion occurs because the sensor cannot see "through" objects; the back of a van or the street behind a parked car is a geometric void.

As noted by Xinge Zhu and colleagues in Shape Signature Networks (SSN), this partial visibility makes it difficult for neural networks to estimate the true center and boundaries of an object. If a detector only sees the front bumper of a car, it may struggle to guess where the trunk ends.

Hallucinating Geometry with Shape Priors

To fill these gaps, models must "know" what objects are supposed to look like. This is often framed as a Point Cloud Completion task.

Early approaches like GRNet, introduced by Haozhe Xie, use gridding and 3D convolutions to regularize sparse points into a dense structure. Other methods, such as CarveNet by Qing Guo, treat the problem like sculpting, "carving" the final shape from a block of space based on visible surfaces.

More recent work leverages external memory. Point-PC, developed by Chuanqi Jiao and Anan Liu, uses causal inference to retrieve "shape priors" from a database of complete 3D models. By finding a similar car or pedestrian in memory, the system can use that template to fill in the occluded regions of the real-world scan.

From Objects to Entire Scenes

While shape completion focuses on individual items, Semantic Scene Completion (SSC) attempts to densify the entire environment. This requires the model to predict the occupancy and semantic label (e.g., "road," "sidewalk," "vegetation") for every voxel in a 3D volume, even those hidden behind walls or cars.

Helin Cao and Sven Behnke have explored using generative models for this task. Their work on DiffSSC uses Denoising Diffusion Probabilistic Models to "refine" a noisy initial guess into a coherent, dense scene. To make this practical for real-time use, Shengyuan Zhang introduced ScoreLiDAR, which distills these complex diffusion models into faster architectures that can run on autonomous vehicle hardware.

Leveraging Time and Collaboration

If a single frame is incomplete, the missing data might be found in the past or from another perspective.

  1. Temporal Fusion: Auto4D, by Bin Yang and Raquel Urtasun, aggregates multiple LiDAR sweeps over time. As a vehicle moves, it sees different sides of an object, allowing the system to stitch together a complete 4D trajectory.
  2. Multi-Agent Collaboration: Qiming Xia and colleagues proposed DOtA, which uses data shared between multiple connected vehicles. One car's occlusion is another car's clear view; by merging these perspectives, the system can detect objects that are completely hidden from any single vehicle's viewpoint.

Why Completion Matters for Perception

The goal of these completion techniques is not just to create "pretty" 3D models, but to improve downstream tasks like detection and tracking.

For instance, 3D IoU-Net demonstrates that by better understanding the full extent of an object, a model can more accurately predict the Intersection-over-Union (IoU) of its bounding boxes. Similarly, DOPS, by Mahyar Najibi and Thomas A. Funkhouser, shows that predicting shape and detecting objects in a single pass leads to more robust performance in complex urban settings. By "filling in the blanks," these systems transform sparse, noisy sensor data into a reliable foundation for autonomous decision-making.

Papers in this version

Showing 60 of 1,183 papers.

History & citationVersion 2 · 2026-10-04

Current · Permanent link

  1. Version 2 (latest) · 2026-10-04 · 1551 papers
  2. Version 1 · 2026-10-04 · 1183 papers
Cite this direction

Latest version (a living page):

@misc{lacuna_geometric_incompleteness_c922205a,
  title = {geometric incompleteness},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/geometric-incompleteness/rdr_c922205a6fb44cae85c731ed7a653bb8},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_reconstructing_complete_3d_geometry_from_c922205a_v1,
  title = {Reconstructing Complete 3D Geometry from Sparse LiDAR Scans},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/reconstructing-complete-3d-geometry-from-sparse-lidar-scans/rdr_c922205a6fb44cae85c731ed7a653bb8/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}