Shape Completion with Learned Geometric Priors

496 concepts370 papers9 proposals

When a sensor captures a 3D scene, it only sees what is directly in its line of sight. Whether utilizing a standard camera, a LiDAR scanner, or millimeter-wave radar, the resulting data is inherently partial. It suffers from self-occlusion (the back of an object is hidden), environmental occlusion (objects blocking one another), and sensor signal dropouts caused by unfavorable surface angles, as seen in specular radar imaging (Valentin Braeutigam et al.).

Traditional geometric reconstruction algorithms, such as standard Truncated Signed Distance Functions (TSDFs), treat space purely locally. They can aggregate visible surfaces but remain fundamentally blind to what lies behind them (John McCormac et al.). To fill in these blanks, a system needs more than just spatial interpolation; it requires an understanding of what it is looking at. It requires a learned geometric prior.

The Shape Manifold as a Search Space

The core intuition behind modern shape completion is to transform reconstruction from an unconstrained generation problem into a constrained search problem. Instead of predicting the positions of thousands of independent 3D points from scratch, models learn a low-dimensional shape manifold—a latent space where every point represents a valid, complete 3D structure belonging to an object category.

Consider the approach introduced by Julian Straub in DeepSDF. By training on large-scale datasets of 3D shapes, a neural network learns to map a continuous Signed Distance Function (SDF) to a compact latent code. When faced with a partial or noisy observation, the system does not try to guess the missing geometry directly. Instead, it optimizes within this learned latent space to find the specific shape code that best aligns with the observed surfaces. Because the latent space only contains valid shapes, the decoded output automatically fills in the unobserved regions with structurally sound, watertight geometry.

Partial Input ───► [ Optimization in Latent Space ] ───► Complete 3D Shape
                          │
                          ▼
               Learned Shape Manifold
             (Restricted to Valid Shapes)

This concept of manifold projection is highly versatile. Benjamin Burchfiel et al. demonstrate this probabilistically with Bayesian Eigenobjects, projecting partial data into category-specific manifolds via Variational Bayesian PCA to infer missing details. Similarly, Diego Rodriguez et al. utilize non-rigid registration in a learned low-dimensional subspace. This allows a system to deform a categorical template to match a single partial point cloud, ensuring the completed geometry adheres to known structural regularities even when major components are entirely occluded.

Continuous Fields vs. Discrete Motifs

How these geometric priors are structured dictates the types of shapes a model can successfully recover. The literature reveals two primary paradigms: continuous implicit fields and discrete structural templates.

Continuous implicit representations, like those in DeepSDF, represent geometry as a field where a network predicts spatial coordinates. This approach enforces a continuous, watertight inductive bias. For instance, Zihan Zhu leverages this in NICE-SLAM by pre-training small decoders on diverse indoor datasets, enabling the system to perform "geometry forecasting" to fill in unobserved gaps in large scenes. In endoscopy, Víctor M. Batlle et al. use signed distance functions in LightNeuS to guarantee continuous surface estimation in regions completely missed by the camera view.

Conversely, other approaches rely on structural anchors or dictionaries of geometric motifs:

  • Global Templates: Fan Duan introduces T-CorresNet, which uses a Gaussian spherical template as a global semantic prior to establish a rough topological anchor before refining local details.
  • Local Motifs: On a local scale, Xin Hu et al. propose a learnable dictionary of geometric prototypes. This allows models to denoise and complete sparse point clouds by referencing stored, pose-invariant motifs rather than relying solely on local point distributions.

Why Category Awareness Matters

A purely geometric prior (like bilateral symmetry or surface smoothness) can only handle minor surface gaps. True robustness to heavy occlusion requires semantic or category-level awareness. When a model knows an object is a "vehicle" or a "chair," it can leverage an internal structural blueprint to resolve severe ambiguities.

Wonbong Jang demonstrates this with CodeNeRF, which disentangles shape and texture across object categories, allowing a neural radiance field to accurately reconstruct the hidden back-side of an object from a single viewpoint. When visual data is highly ambiguous or textureless, incorporating category-level semantic knowledge acts as a powerful regularizer, preventing the generation of physically impossible shapes (Sid Ying-Ze Bao et al.).

Even when real-world training data lacks full 3D ground truth, weakly-supervised frameworks like the one proposed by Andreas Geiger et al. can leverage synthetic shape priors to successfully constrain and complete partial real-world scans.

Generalization and Downstream Utility

Predicting unseen geometry is not merely about creating visually appealing models; learned geometric priors fundamentally improve downstream tasks like tracking, mapping, and robotic interaction.

In autonomous driving, forcing a network to perform shape completion as an auxiliary task forces it to learn richer latent representations. This significantly improves its ability to distinguish target vehicles from background clutter under heavy occlusion. In robotics, completing the geometry of an occluded object allows for functional grasp planning; a robot can infer the location and shape of a hidden handle or tool component by aligning its partial view with the category's latent space (Diego Rodriguez et al.).

Crucially, these models do not merely memorize training instances. As shown by David Held et al. in PCN: Point Completion Network, deep learning architectures can internalize fundamental geometric principles that generalize even to object categories unseen during training, enabling coherent reconstructions from extremely sparse inputs like low-resolution LiDAR scans.

History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_learned_geometric_priors_6987e00b,
  title = {learned geometric priors},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/learned-geometric-priors/rdr_6987e00b42334d3da30bc98706950b24},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 1, frozen text):

@misc{lacuna_learned_geometric_priors_6987e00b_v1,
  title = {learned geometric priors},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/learned-geometric-priors/rdr_6987e00b42334d3da30bc98706950b24/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake