GridPull: Towards Scalability in Learning Implicit Representations from 3D Point Clouds

Chao Chen, Yu-Shen Liu, Zhizhong Han

Abstract

Surface reconstruction from 3D point clouds is a critical task in computer vision, recently dominated by neural implicit representations. However, these methods often struggle with scalability due to the slow convergence of neural networks and high computational costs. We present GridPull, a "neural-free" approach that optimizes a discrete distance field directly on a 3D grid. By utilizing a specialized "pulling" loss and Total Variation regularization, GridPull achieves state-of-the-art accuracy while being significantly faster and more scalable than traditional neural methods, capable of processing point clouds with millions of points.

Introduction

Reconstructing smooth, high-fidelity surfaces from raw 3D point clouds is fundamental for applications in robotics, AR/VR, and digital twins. While neural implicit representations—which overfit a network to represent a distance or occupancy field—have set new benchmarks for quality, they face a significant bottleneck: scalability. The computational cost of calculating distances for thousands of query points against millions of surface points in every training iteration makes these methods impractical for large-scale scenes or dense scans. GridPull addresses this by replacing the neural network with a directly optimized grid-based representation.

Methods

GridPull introduces a discrete approach to learning implicit representations, where the learnable parameters are signed distance values at the vertices of a uniform 3D grid.

Discrete Distance Field and Pulling Loss

Instead of a network forward pass, the distance at any query point $q$ is computed via fast trilinear interpolation of the eight nearest grid vertices. To align this field with the input point cloud, GridPull employs a Pulling Loss. For a sampled query point $q$, the current field's gradient is used to "pull" $q$ toward the surface, resulting in a projected point $p$. The loss minimizes the distance between $p$ and its actual nearest neighbor in the point cloud.

Regularization and Efficiency

Because discrete grids lack the inherent smoothness of neural networks, GridPull incorporates a Total Variation (TV) Loss. This encourages neighboring grid vertices to maintain consistent values, ensuring a continuous and smooth field. To further boost efficiency, the method uses Bandwidth Optimization, updating only grid cells within a specific distance of the surface. Lightning-fast nearest neighbor searches are facilitated by a KD-tree.

Overview of the GridPull pipeline and its various loss functions.

Results and Discussion

GridPull was evaluated across diverse benchmarks, including object-level datasets (ShapeNet, FAMOUS, Thingi10k) and large-scale scene datasets (Matterport3D, KITTI).

Performance and Scalability

GridPull consistently outperforms recent neural-based methods like NeuralPull (NP) and Predictive Context Prior (PCP) in both speed and accuracy. On the FAMOUS dataset, GridPull achieved a mean reconstruction time of 201 seconds, significantly faster than NP (605s) and PCP (>3500s). Notably, GridPull successfully reconstructed a KITTI dataset frame containing 13.8 million points in approximately one hour—a task typically impossible for standard neural implicit methods.

Visual Fidelity

Visual comparisons on ShapeNet demonstrate GridPull's superior ability to capture fine geometric details and thin structures that are often smoothed over or missed by neural counterparts.

Visual comparison on ShapeNet showing GridPull's ability to capture fine details compared to other methods.

Conclusion

GridPull demonstrates that neural networks are not strictly mandatory for high-quality implicit surface reconstruction. By leveraging well-regularized discrete grids, the method achieves superior speed and detail. While performance is currently tied to grid resolution, future work involving adaptive structures like Octrees could further reduce memory overhead, pushing the boundaries of large-scale 3D reconstruction.

Summary updated

Abstract

Learning implicit representations has been a widely used solution for surface reconstruction from 3D point clouds. The latest methods infer a distance or occupancy field by overfitting a neural network on a single point cloud. However, these methods suffer from a slow inference due to the slow convergence of neural networks and the extensive calculation of distances to surface points, which limits them to small scale points. To resolve the scalability issue in surface reconstruction, we propose GridPull to improve the efficiency of learning implicit representations from large scale point clouds. Our novelty lies in the fast inference of a discrete distance field defined on grids without using any neural components. To remedy the lack of continuousness brought by neural networks, we introduce a loss function to encourage continuous distances and consistent gradients in the field during pulling queries onto the surface in grids near to the surface. We use uniform grids for a fast grid search to localize sampled queries, and organize surface points in a tree structure to speed up the calculation of distances to the surface. We do not rely on learning priors or normal supervision during optimization, and achieve superiority over the latest methods in terms of complexity and accuracy. We evaluate our method on shape and scene benchmarks, and report numerical and visual comparisons with the latest methods to justify our effectiveness and superiority. The code is available at https://github.com/chenchao15/GridPull.
BibTeX
@inproceedings{chen2023gridpull,
  author = {Chao Chen and Yu-Shen Liu and Zhizhong Han},
  title = {{GridPull: Towards Scalability in Learning Implicit Representations from 3D Point Clouds}},
  booktitle = {IEEE/CVF International Conference on Computer Vision},
  year = {2023},
}
Figure