Abstract
In this paper, we propose Neural Points, a novel point cloud representation and apply it to the arbitrary-factored upsampling task. Different from traditional point cloud representation where each point only represents a position or a local plane in the 3D space, each point in Neural Points represents a local continuous geometric shape via neural fields. Therefore, Neural Points contain more shape information and thus have a stronger representation ability. Neural Points is trained with surface containing rich geometric details, such that the trained model has enough expression ability for various shapes. Specifically, we extract deep local features on the points and construct neural fields through the local isomorphism between the 2D parametric domain and the 3D local patch. In the final, local neural fields are integrated together to form the global surface. Experimental results show that Neural Points has powerful representation ability and demonstrate excellent robustness and generalization ability. With Neural Points, we can resample point cloud with arbitrary resolutions, and it outperforms the state-of-the-art point cloud upsampling methods.
Introduction
Point clouds are a fundamental data structure for 3D computer vision, yet their discrete nature poses significant challenges. Standard representations treat points as isolated $(x, y, z)$ coordinates, which leads to a loss of surface continuity when data is sparse or when zooming into fine details. Traditional upsampling methods—the process of increasing point density—often operate in a discrete-to-discrete fashion, producing a fixed number of output points (e.g., $4\times$ or $16\times$ the input). These methods frequently struggle with complex geometries, noise, and the inability to scale to arbitrary resolutions.
Neural Points introduces a paradigm shift by representing each point as the center of a local, continuous surface patch defined by a neural field. This approach enables "arbitrary-factored upsampling," allowing for the generation of high-resolution, smooth surfaces at any scale from sparse input data.
The Neural Points Representation
The core innovation of this work is the transition from static coordinates to dynamic local features that parameterize neural fields. Instead of a point being a simple dot, it becomes a "Neural Point" that encodes the local geometry of its neighborhood.
Methodology
The construction of Neural Points involves several key technical steps:
- Local Isomorphism: Leveraging the topological property that small 3D manifold patches are isomorphic to 2D disks, the authors establish a bijective mapping between a 2D parametric domain and the 3D local patch.
- Neural Field Network: A shared Multi-Layer Perceptron (MLP) represents this mapping. It takes a 2D coordinate and a local feature vector as input to predict the corresponding 3D position on the surface.
- Feature Extraction: A Dynamic Graph CNN (DGCNN) backbone is utilized to extract deep local features, ensuring that each Neural Point captures essential geometric information like curvature and edges.
- Global Integration: To handle overlapping patches, a distance-based weighting scheme "stitches" local neural fields into a globally consistent continuous surface.
- Projection-based Loss: Training is guided by a projection strategy that pulls points toward the underlying continuous surface, avoiding the noise sensitivity of standard metrics like Chamfer distance.
Experimental Results
The authors evaluated Neural Points on diverse datasets, including high-detail models from Sketchfab, the PU-GAN dataset, and real-world data from iPhone depth sensors and LiDAR.
Performance and Visual Quality
Neural Points achieves state-of-the-art (SOTA) performance across standard metrics, including Chamfer Distance (CD), Hausdorff Distance (HD), and Point-to-Surface (P2F) distance. Unlike previous methods that often produce "clumpy" or noisy distributions at high upsampling factors, Neural Points maintains uniform and smooth surface reconstructions.
The model demonstrates remarkable robustness to noise and excellent generalization to real-world data. Furthermore, its resolution-independent nature allows for extreme upsampling (up to $256\times$), reconstructing coherent shapes from inputs that are nearly unrecognizable.
Conclusion
Neural Points provides a powerful and memory-efficient representation for 3D point clouds. By treating points as local neural fields, it overcomes the limitations of discrete representations and fixed-factor upsampling. While currently focused on geometry, the entire trained model is only about 2.53 MB, making it highly practical. Future work aims to incorporate multi-modal data like color and global semantic context to further enhance its expressive power.
Summary updated