Arbitrary-Scale Point Cloud Upsampling

35 concepts22 papers

The Bottleneck of Fixed-Scale Upsampling

Traditional deep learning architectures for 3D point cloud densification operate under a rigid constraint: they are hard-coded to upsample data by fixed, integer factors (such as $2\times$, $4\times$, or $16\times$). As noted by Le Hui et al. in Progressive Point Cloud Deconvolution Generation Network and Shi Qiu et al. in PU-Transformer: Point Cloud Upsampling Transformer, these models rely on discrete point-splitting layers or rigid deconvolutional blocks. This architectural dependency makes it impossible to dynamically adjust point density based on local geometric complexity or to achieve non-integer resolution increases.

In real-world applications, such as processing LiDAR scans from autonomous vehicles or handling varying sensor resolutions, this lack of flexibility is a major bottleneck. If a system requires an arbitrary or non-standard target density, developers are typically forced to design and train separate scale-specific networks or chain multiple fixed-scale models together in a multi-stage pipeline. The authors of Self-Supervised Arbitrary-Scale Point Clouds Upsampling via Implicit Neural Representation point out that such multi-stage chaining is computationally expensive and prone to accumulating geometric errors.

To overcome these limitations, a new paradigm has emerged: arbitrary-scale point cloud upsampling. The goal is to train a single unified model capable of densifying a sparse 3D input to any desired real-number magnification factor on the fly during inference.

Decoupling Point Generation from Geometry

The core insight behind arbitrary-scale upsampling is the separation of point generation (determining how many points to create) from geometric refinement (determining where those points should precisely sit). By removing the target density from the network's internal layer dimensions, the architecture becomes scale-agnostic.

One intuitive way to achieve this is through geometric interpolation followed by learned refinement. In Grad-PU: Arbitrary-Scale Point Cloud Upsampling via Gradient Descent with Learned Distance Functions, Yun He et al. demonstrate that initial candidate points can be generated using simple, non-learned Euclidean midpoint interpolation between neighboring points. Because this step is purely algorithmic, it can generate any arbitrary number of points. The neural network's role is shifted entirely to location refinement: it learns a continuous distance function and uses gradient descent to pull these raw candidate points onto the true underlying 3D surface, eliminating artifacts like outliers or surface shrinkage.

Other structural strategies focus on flexible coordinate mappings and deformations:

Continuous Manifolds via Implicit Neural Representations

The most prominent and mathematically rigorous mechanism for achieving resolution independence is shifting from discrete point representations to Implicit Neural Representations (INRs) or Neural Fields. Instead of training a network to output a discrete matrix of 3D coordinates, the network is trained to approximate a continuous mathematical surface.

Discrete Input        Continuous Implicit Field        Arbitrary-Scale Output
  (Points)             (Learned Distance Function)        (Any Density Target)
   •   •                  ┌─────────────────┐                 • • • • • •
     •       ───>         │  f(x, y, z) = 0 │      ───>      • • • • • • •
   •   •                  └─────────────────┘                 • • • • • •

In Neural Points: Point Cloud Representation with Neural Fields for Arbitrary Upsampling, Wanquan Feng et al. propose treating each input point as the center of a local, continuous surface patch represented by a neural field. Because a neural field can be queried at any arbitrary coordinate, the representation itself becomes resolution-independent.

When geometry is modeled as a continuous field, upsampling transforms from a difficult generation problem into a straightforward sampling and projection task. As explained by Shujuan Li and Baorui Ma in Learning Continuous Implicit Field with Local Distance Indicator for Arbitrary-Scale Point Cloud Upsampling, the network can learn a global Unsigned Distance Field (UDF). To upsample, the system samples a desired number of query points in space and projects them onto the learned implicit surface. Since the underlying function is continuous, the density of the output can be increased indefinitely without retraining the model.

This continuous formulation provides powerful advantages across downstream tasks:

Meta-Learning and Generative Diffusion Fields

Beyond geometric interpolation and implicit fields, researchers have introduced meta-learning and generative frameworks to dynamically adjust to arbitrary scales.

Shuquan Ye approaches the problem via weight modulation in Meta-PU: An Arbitrary-Scale Upsampling Network for Point Cloud. Instead of forcing a static network to output different densities, the author employs a meta-learning subnetwork that dynamically predicts and adjusts the internal weights of the main upsampling network based on the specific magnification factor requested at inference time.

Alternatively, generative diffusion models offer a probabilistic solution to scale flexibility. Because diffusion models conceptualize generation as an iterative denoising process, they are not bound by rigid layer dimensions. Xianjing Cheng et al. (PVNet: Point-Voxel Interaction LiDAR Scene Upsampling Via Diffusion Models) and Yi Du et al. (SuperPC: A Single Diffusion Model for Point Cloud Completion, Upsampling, Denoising, and Colorization) demonstrate that generative diffusion frameworks naturally support continuous-scale upsampling. By conditioning the diffusion process on sparse geometric features, these models can stochastically generate and refine an arbitrary number of points, providing the flexibility needed to handle diverse sensor configurations and varying real-world output requirements within a single architecture.

Key Foundational Work

Progressive Point Cloud Deconvolution Generation Network (2020) — contributed 1 concept elements to this direction.
A Simple Deconvolutional Mechanism for Point Clouds and Sparse Unordered Data (Student Abstract) (2020) — contributed 1 concept elements to this direction.
GraphX-Convolution for Point Cloud Deformation in 2D-to-3D Conversion (2019) — contributed 1 concept elements to this direction.
History & citationVersion 2 · 2026-10-04

Current · Permanent link

  1. Version 2 (latest) · 2026-10-04 · 21 papers
  2. Version 1 · 2026-10-04 · 20 papers
Cite this direction

Latest version (a living page):

@misc{lacuna_arbitrary_scale_upsampling_0e038b0a,
  title = {arbitrary-scale upsampling},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/arbitrary-scale-upsampling/rdr_0e038b0aaaa946d99cbd904ead4a2409},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 2, frozen text):

@misc{lacuna_arbitrary_scale_upsampling_0e038b0a_v2,
  title = {arbitrary-scale upsampling},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/arbitrary-scale-upsampling/rdr_0e038b0aaaa946d99cbd904ead4a2409/v/2},
  note = {Version 2, frozen 2026-10-04. Accessed 2026-11-20},
}

See something wrong? Report a mistake