The Bottleneck of Fixed-Scale Upsampling
Traditional deep learning architectures for 3D point cloud densification operate under a rigid constraint: they are hard-coded to upsample data by fixed, integer factors (such as $2\times$, $4\times$, or $16\times$). As noted by Le Hui et al. in Progressive Point Cloud Deconvolution Generation Network and Shi Qiu et al. in PU-Transformer: Point Cloud Upsampling Transformer, these models rely on discrete point-splitting layers or rigid deconvolutional blocks. This architectural dependency makes it impossible to dynamically adjust point density based on local geometric complexity or to achieve non-integer resolution increases.
In real-world applications, such as processing LiDAR scans from autonomous vehicles or handling varying sensor resolutions, this lack of flexibility is a major bottleneck. If a system requires an arbitrary or non-standard target density, developers are typically forced to design and train separate scale-specific networks or chain multiple fixed-scale models together in a multi-stage pipeline. The authors of Self-Supervised Arbitrary-Scale Point Clouds Upsampling via Implicit Neural Representation point out that such multi-stage chaining is computationally expensive and prone to accumulating geometric errors.
To overcome these limitations, a new paradigm has emerged: arbitrary-scale point cloud upsampling. The goal is to train a single unified model capable of densifying a sparse 3D input to any desired real-number magnification factor on the fly during inference.
Decoupling Point Generation from Geometry
The core insight behind arbitrary-scale upsampling is the separation of point generation (determining how many points to create) from geometric refinement (determining where those points should precisely sit). By removing the target density from the network's internal layer dimensions, the architecture becomes scale-agnostic.
One intuitive way to achieve this is through geometric interpolation followed by learned refinement. In Grad-PU: Arbitrary-Scale Point Cloud Upsampling via Gradient Descent with Learned Distance Functions, Yun He et al. demonstrate that initial candidate points can be generated using simple, non-learned Euclidean midpoint interpolation between neighboring points. Because this step is purely algorithmic, it can generate any arbitrary number of points. The neural network's role is shifted entirely to location refinement: it learns a continuous distance function and uses gradient descent to pull these raw candidate points onto the true underlying 3D surface, eliminating artifacts like outliers or surface shrinkage.
Other structural strategies focus on flexible coordinate mappings and deformations:
- Affine Combinations: Luqing Luo et al. introduce PU-EVA: An Edge-Vector based Approximation Solution for Flexible-scale Point Cloud Upsampling, which uses edge-vector-based affine combinations to generate new points. This completely bypasses fixed-rate feature expansion layers and allows flexible-scale densification in a single training pass.
- Kernel Resampling: In RepKPU: Point Cloud Upsampling with Kernel Point Representation and Deformation, Yi Rong et al. utilize adaptive kernel points and cross-attention guided deformations. Their resampling strategy allows a model trained on a fixed upsampling rate to generalize seamlessly to arbitrary output scales during inference.
- Local Tangent Planes: Hao Pan et al. develop iPUNet: Iterative Cross Field Guided Point Cloud Upsampling, which maps coordinates from a continuous 2D local tangent plane to 3D spatial offsets, effectively decoupling the output density from the underlying network architecture.
Continuous Manifolds via Implicit Neural Representations
The most prominent and mathematically rigorous mechanism for achieving resolution independence is shifting from discrete point representations to Implicit Neural Representations (INRs) or Neural Fields. Instead of training a network to output a discrete matrix of 3D coordinates, the network is trained to approximate a continuous mathematical surface.
Discrete Input Continuous Implicit Field Arbitrary-Scale Output
(Points) (Learned Distance Function) (Any Density Target)
• • ┌─────────────────┐ • • • • • •
• ───> │ f(x, y, z) = 0 │ ───> • • • • • • •
• • └─────────────────┘ • • • • • •
In Neural Points: Point Cloud Representation with Neural Fields for Arbitrary Upsampling, Wanquan Feng et al. propose treating each input point as the center of a local, continuous surface patch represented by a neural field. Because a neural field can be queried at any arbitrary coordinate, the representation itself becomes resolution-independent.
When geometry is modeled as a continuous field, upsampling transforms from a difficult generation problem into a straightforward sampling and projection task. As explained by Shujuan Li and Baorui Ma in Learning Continuous Implicit Field with Local Distance Indicator for Arbitrary-Scale Point Cloud Upsampling, the network can learn a global Unsigned Distance Field (UDF). To upsample, the system samples a desired number of query points in space and projects them onto the learned implicit surface. Since the underlying function is continuous, the density of the output can be increased indefinitely without retraining the model.
This continuous formulation provides powerful advantages across downstream tasks:
- Self-Supervised Learning: Wenbo Zhao et al. demonstrate in Self-Supervised Arbitrary-Scale Point Clouds Upsampling via Implicit Neural Representation that by over-sampling seed points near an estimated continuous surface and applying Farthest Point Sampling, a network can match supervised performance without requiring dense ground-truth data for training.
- Surface Estimation: In PLATYPUS: Progressive Local Surface Estimator for Arbitrary-Scale Point Cloud Upsampling, Donghyun Kim and Yumin Kim combine continuous distance fields with a curvature-based sampling strategy to preserve intricate details in high-curvature regions at any scale.
- Data Compression: Hao Xu 0051 et al. apply this concept to 3D data storage in Fast Point Cloud Geometry Compression with Context-Based Residual Coding and INR-Based Refinement. By decoding compressed data into a continuous mathematical surface rather than a fixed set of points, the decoder can reconstruct the underlying 3D object at any desired point density on demand.
Meta-Learning and Generative Diffusion Fields
Beyond geometric interpolation and implicit fields, researchers have introduced meta-learning and generative frameworks to dynamically adjust to arbitrary scales.
Shuquan Ye approaches the problem via weight modulation in Meta-PU: An Arbitrary-Scale Upsampling Network for Point Cloud. Instead of forcing a static network to output different densities, the author employs a meta-learning subnetwork that dynamically predicts and adjusts the internal weights of the main upsampling network based on the specific magnification factor requested at inference time.
Alternatively, generative diffusion models offer a probabilistic solution to scale flexibility. Because diffusion models conceptualize generation as an iterative denoising process, they are not bound by rigid layer dimensions. Xianjing Cheng et al. (PVNet: Point-Voxel Interaction LiDAR Scene Upsampling Via Diffusion Models) and Yi Du et al. (SuperPC: A Single Diffusion Model for Point Cloud Completion, Upsampling, Denoising, and Colorization) demonstrate that generative diffusion frameworks naturally support continuous-scale upsampling. By conditioning the diffusion process on sparse geometric features, these models can stochastically generate and refine an arbitrary number of points, providing the flexibility needed to handle diverse sensor configurations and varying real-world output requirements within a single architecture.
Key Foundational Work
Progressive Point Cloud Deconvolution Generation Network (2020) — contributed 1 concept elements to this direction.
A Simple Deconvolutional Mechanism for Point Clouds and Sparse Unordered Data (Student Abstract) (2020) — contributed 1 concept elements to this direction.
GraphX-Convolution for Point Cloud Deformation in 2D-to-3D Conversion (2019) — contributed 1 concept elements to this direction.