Abstract
Point completion refers to completing the missing geometries of an object from incomplete observations. Main-stream methods predict the missing shapes by decoding a global feature learned from the input point cloud, which often leads to deficient results in preserving topology consistency and surface details. In this work, we present ME-PCN, a point completion network that leverages 'emptiness' in 3D shape space. Given a single depth scan, previous methods often encode the occupied partial shapes while ignoring the empty regions (e.g. holes) in depth maps. In contrast, we argue that these 'emptiness' clues indicate shape boundaries that can be used to improve topology representation and detail granularity on surfaces. Specifically, our ME-PCN encodes both the occupied point cloud and the neighboring 'empty points'. It estimates coarse-grained but complete and reasonable surface points in the first stage, followed by a refinement stage to produce fine-grained surface details. Comprehensive experiments verify that our ME-PCN presents better qualitative and quantitative performance against the state-of-the-art. Besides, we further prove that our 'emptiness' design is lightweight and easy to embed in existing methods, which shows consistent effectiveness in improving the CD and EMD scores.
Introduction
Capturing 3D data with LiDAR or depth cameras usually results in "partial scans"—point clouds that are sparse and missing geometry due to occlusions or sensor limitations. Point completion aims to fill in these gaps to create a watertight, high-resolution model.
The standard approach involves an encoder-decoder architecture: an encoder (like PointNet) compresses the partial input into a global feature vector, and a decoder generates the missing points. However, this often leads to "over-smoothed" results. Specifically, these models struggle with topology consistency—they often accidentally fill in holes (like the space between chair slats) or fail to capture sharp boundaries because they only look at where points are, ignoring the valuable information of where points cannot be.
Methodology
The core innovation of ME-PCN is leveraging Mask Emptiness. When a depth camera captures a scene, any pixel in the 2D mask that doesn't contain an object indicates a clear line of sight through 3D space.
As shown in the architecture overview, the process involves several key stages:
- Ray Generation and Sampling: The authors back-project "empty" pixels from the 2D mask into 3D rays. They sample "empty points" along these rays that are in the neighborhood of the observed partial shape.
- Dual Encoding: Instead of a single encoder, ME-PCN uses two separate branches. One encodes the occupied partial point cloud, and the other encodes the sampled "empty points" (including their ray directions and distances). These are concatenated into a global feature vector that understands both the object's presence and its boundaries.
- Coarse-to-Fine Decoding with Ray Resampling:
- Stage 1: A coarse decoder predicts a rough, complete shape.
- Stage 2: The model "re-queries" the emptiness information. It checks if the predicted coarse points fall into regions known to be empty. This feedback loop allows a refinement stage to produce fine-grained surface details and ensure holes remain open.
The model is trained using Earth Mover’s Distance (EMD) and an expansion regularizer to ensure an even distribution of points across the completed surface.
Experiments and Results
The researchers evaluated their method on a subset of the ShapeNet dataset, focusing on 14 categories (e.g., chairs, tables, lamps, faucets) comprising nearly 30,000 CAD models. They also tested the model's generalization on real-world scans, demonstrating that the "emptiness" logic holds up even outside of synthetic training environments.
ME-PCN achieved top-tier results across several metrics. On the ShapeNet benchmark, ME-PCN achieved an average EMD of 5.93 (lower is better), compared to 6.99 for the previous leader, MSN. It also showed superior performance in Chamfer Distance (CD).
Visual comparisons show that ME-PCN is significantly better at maintaining the thin structures of chair legs and the distinct gaps in slatted furniture, whereas other models tend to produce "blobby" or filled-in reconstructions. Furthermore, the authors simulated noisy masks and found that the model's performance degraded by less than 3%, proving the approach is practical for real-world sensors.
Conclusion
ME-PCN demonstrates that "emptiness" is a powerful cue for defining 3D boundaries and preserving topology. By explicitly telling the network where the "empty" space is, the authors provide a much stronger signal for where the surface must end. The design is lightweight and modular, making it easy to embed in existing methods. Future work could involve integrating this logic with implicit function representations or exploring temporal "emptiness" clues from video sequences.
Summary updated