Abstract
In this paper, we propose Neural Points, a novel point cloud representation and apply it to the arbitrary-factored upsampling task. Different from traditional point cloud representation where each point only represents a position or a local plane in the 3D space, each point in Neural Points represents a local continuous geometric shape via neural fields. Therefore, Neural Points contain more shape information and thus have a stronger representation ability. Neural Points is trained with surface containing rich geometric details, such that the trained model has enough expression ability for various shapes. Specifically, we extract deep local features on the points and construct neural fields through the local isomorphism between the 2D parametric domain and the 3D local patch. In the final, local neural fields are integrated together to form the global surface. Experimental results show that Neural Points has powerful representation ability and demonstrate excellent robustness and generalization ability. With Neural Points, we can resample point cloud with arbitrary resolutions, and it outperforms the state-of-the-art point cloud upsampling methods.
Introduction
Point clouds serve as the fundamental data format for 3D sensors such as LiDAR and depth cameras. However, their discrete nature presents significant limitations for high-fidelity geometric modeling. Traditional representations treat point clouds as a collection of independent $(x, y, z)$ coordinates, which restricts their utility in tasks like upsampling. Most existing upsampling methods follow a "discrete-to-discrete" strategy, which often requires training specific models for fixed upsampling factors (e.g., $4\times$ or $16\times$) and frequently results in noisy or "clumpy" surfaces in high-detail areas.
Neural Points addresses these challenges by introducing a paradigm shift: representing each point not as a static coordinate, but as a local, continuous mathematical function—a neural field. This approach enables "resolution-free" geometry, allowing researchers to upsample sparse point clouds to any arbitrary density with state-of-the-art precision and surface smoothness.
Methods
The core of the Neural Points representation is the combination of local feature learning with implicit neural functions. The pipeline consists of three main stages:
- Local Feature Extraction: For every point in a sparse input cloud, the system uses a Dynamic Graph CNN (DGCNN) backbone to extract a deep feature vector. This vector encodes the local geometric context, such as curves and edges, around that point.
- Local Neural Fields: Instead of predicting new points directly, the model learns a mapping (an MLP) between a 2D parametric domain (a flat disk) and a 3D local surface patch. This mapping leverages the local isomorphism between the 2D domain and the 3D manifold. Because this mapping is a continuous function, one can sample an infinite number of coordinates from the 2D disk and "project" them into 3D space.
- Neural Fields Integration: Since these local patches overlap, the authors implement a "partition of unity" style integration strategy. They use a weighted sum to blend overlapping patches into a single, globally consistent continuous surface.
Experiments and Results
The authors evaluated Neural Points against several benchmarks, including the high-detail Sketchfab dataset and the PU-GAN dataset. To prove robustness, they also tested on raw scans from an iPhone X depth sensor and KITTI LiDAR data, which are notoriously noisy and incomplete.
Performance and Visual Quality
Neural Points outperformed existing state-of-the-art (SOTA) methods, including PU-Net, PU-GAN, and PUGeo-Net, across major metrics such as Chamfer Distance (CD) and Hausdorff Distance (HD). Qualitatively, the results are significantly smoother. While other methods often produce "clumping" or "holes" at high upsampling factors (like $256\times$), Neural Points maintains a clean, manifold-like surface.
Key Insights
- Local vs. Global Fields: By using many small, local fields rather than one giant global field, the model is better at capturing fine-grained details and generalizes effectively to entirely new shapes.
- Resolution-Free Inference: Because the representation is continuous, there is no need to retrain the model to change the upsampling factor. Dense sampling can be performed at inference time by simply increasing the sampling density in the 2D domain.
- Efficiency and Robustness: The model is remarkably lightweight, with a total storage overhead of only 2.53 MB. Furthermore, it handles noisy real-world scans significantly better than previous methods, which often struggle with the "bumpy" nature of raw sensor data.
Conclusion
Neural Points provides a robust and flexible framework for representing 3D geometry. While currently focused on geometric shape, future iterations could incorporate other modalities like color or material properties. Additionally, integrating global semantic information could further improve reconstruction in extremely sparse or occluded areas. Despite these current limitations, Neural Points sets a new benchmark for point cloud upsampling and continuous surface representation.
Summary updated