A depth sensor or LiDAR scanner returns a cloud of 3D points, each displaced from the true surface by measurement noise. Current learned denoisers move every point to one "best" location and give no sign of how sure they are. For a distant car or a thin chair leg, the cleaned surface may be partly invented. Image restoration handles this by drawing samples from the posterior, the distribution of clean signals consistent with the observation, and reading off per-pixel error bars. That recipe does not carry over to 3D, because point clouds have no fixed ordering. Two sampled clean clouds have no natural point-to-point correspondence, so "the variance at point i" is undefined. FoldDiff makes this precise: the identity linking a denoiser's residual to the data's score (the gradient of log-density) holds only when coordinates sit in a fixed, known order.
Our observation is that denoising, unlike generation or completion, comes with an ordering for free. Each noisy point is a measurement of some clean surface point, so a posterior can be attached to each observed point: where is the surface point that produced this measurement? We propose to learn this per-observation posterior with variational inference, meaning we fit a tractable distribution to the true posterior by optimization. The variational family is chosen to match the shape this posterior actually has, which is a thin curved sheet hugging the surface and not a round blob. We train it with a mass-covering objective so that it does not understate uncertainty. The result is a single-pass denoiser that returns calibrated per-point credible intervals along the surface normal. Because its point estimate is a surface mode and not a Euclidean mean, it does not shrink the shape.
Core Claim
For point cloud denoising, the per-point posterior $p(x_i \mid y)$ has two properties. Here $y=\{y_j\}$ is the noisy cloud and $x_i$ is the clean surface point that produced observation $y_i$. First, the posterior is well defined despite permutation ambiguity, because the index $i$ is inherited from the observation. Second, it is strongly non-Gaussian. Near curved regions it concentrates on a curved patch of surface, so its Euclidean mean lies inside the shape.
This second property explains a known failure mode. "Point shrinkage" is the tendency of displacement regression to pull shapes inward, reported by Score-Based Point Cloud Denoising and Differentiable Manifold Reconstruction. It is the 3D counterpart of the blur that image restoration attributes to the conditional mean, as analyzed by Rethinking Lossy Compression: The Rate-Distortion-Perception Tradeoff. It follows that a Gaussian approximate posterior, or a heteroscedastic regressor that predicts a mean plus a variance, will both shrink the shape and miscalibrate its uncertainty.
We claim that a semi-implicit variational family, trained by forward-KL amortized inference against oracle-surface posteriors, will give three things. The family is a mixture of flat, surface-aligned components whose parameters are coupled across neighboring points. Its outputs will be:
- per-point normal-direction intervals whose conditional coverage holds across density and curvature strata;
- point estimates at least as accurate as mode-seeking score denoisers;
- all of this in one forward pass.
Relation to Prior Work
Two point-cloud denoisers motivate the proposal, because each solves part of the problem and discards uncertainty. Score-Based Point Cloud Denoising learns the score of the noise-convolved surface density with a graph network and climbs to its modes by gradient ascent. This avoids shrinkage, but by construction it returns a single mode and nothing about spread. Diffusion Bridges for 3D Point Cloud Denoising learns a transport path between noisy and clean distributions in 3–5 steps. It is a sampler of whole clouds, not a model of per-point posteriors, and it does not address calibration.
Two variational point-cloud models constrain the design. VRCNet uses a single Gaussian latent for completion and then refines deterministically. LION couples a hierarchical VAE with latent diffusion for unconditional generation. Both place uncertainty in a global shape code, not at the level of individual measurements, and neither is evaluated for calibration.
The method draws on two sources. The first is semi-implicit variational inference as used in Semi-Implicit Graph Variational Auto-Encoders. There, noise injected through a graph encoder produces multimodal, neighborhood-shared posteriors over node embeddings on discrete adjacency graphs. We move this to continuous 3D positions on a k-nearest-neighbor graph, which that paper does not handle. The second is the image-side idea from Stochastic Solutions for Linear Inverse Problems using the Prior Implicit in a Denoiser and Traversing Distortion-Perception Tradeoff: posterior samples give uncertainty, and the low-distortion end of the tradeoff is the conditional mean. That second point is exactly what fails in 3D. We replace the mean with a surface-mode estimate. We do not claim to resolve the full 3D distortion-perception tradeoff.
Research Program
The target distribution. For a training pair of a noisy cloud $y$ and clean surface $S$, taken from a mesh or a dense reference scan, the oracle-surface posterior for observation $i$ is $$p(x_i \mid y_i, S) \propto \mathbf{1}[x_i \in S]\,\ell(y_i \mid x_i).$$ Here $\ell$ is the sensor noise likelihood. It is Gaussian with known $\sigma$ on synthetic data. On real data it is a Student-t distribution whose scale is fit on held-out pairs. We compute this posterior by dense sampling of $S$ near $y_i$.
The training objective. We train an amortized network $q_\phi(x_i \mid y)$ by minimizing $\mathbb{E}_{(y,S)}\,\mathrm{KL}\big(p(x_i \mid y_i,S)\,\|\,q_\phi(x_i \mid y)\big)$. At test time $S$ is unknown. Averaging over surfaces consistent with $y$ makes the minimizer of this objective the true posterior $p(x_i \mid y)=\mathbb{E}_{S\mid y}\,p(x_i\mid y_i,S)$. The reason is that forward KL to a mixture is minimized by that mixture. This forward, mass-covering direction is deliberate. It targets the variance underestimation that reverse-KL evidence-lower-bound training is known for, and we ablate the choice.
The variational family. Each point receives a mixture of $K$ anisotropic Gaussian "pancakes." Each component has a center, a tangent frame, a thin normal variance, and a wide tangential variance. The mixture weights and parameters are produced by a graph network, and a per-neighborhood noise variable is injected semi-implicitly. Neighboring points therefore share uncertainty: a missing-data region widens intervals coherently rather than point by point. Where the semi-implicit layer makes $\log q$ intractable, we use the Monte Carlo surrogate bound from the semi-implicit inference literature. The finite-mixture ablation keeps it exact.
Outputs. The model produces three things:
- the point estimate, which is the highest-density component center (a surface mode);
- the normal-direction credible interval, read from the mixture's marginal along the estimated normal;
- a flagging score equal to the tangential-plus-normal posterior mass outside a tolerance, used to mark possibly invented geometry.
Analysis. On synthetic shapes, the oracle posterior is exactly computable when the surface is known. This lets us separate the error of the variational family (distance from $q$ to the oracle given the true $S$) from the amortization gap. We measure the amortization gap against per-instance Langevin sampling, described below.
Baselines
The naive reference point outputs the noisy point itself as the estimate. It attaches a constant-width normal interval, set on a calibration split so that marginal coverage is exactly 90%. It is not a straw man. It achieves perfect average coverage by construction, so our method beats it only if its coverage is roughly equal within density, range, and curvature strata while its intervals are narrower.
The strongest point-estimate competitor is the gradient-ascent score denoiser of Score-Based Point Cloud Denoising. It reports a Chamfer Distance of 4.708 on the PU-Net dataset at 3% Gaussian noise, against 13.067 for the displacement regressor PCNet. We retrain it on our exact splits with its published step schedule, then tune its step size on the same validation budget we give our model. For uncertainty, we pair it with a five-member deep ensemble trained from different seeds. Per-point spread is defined because each ensemble member moves the same indexed points.
We also evaluate Diffusion Bridges for 3D Point Cloud Denoising on ScanNet++ and ARKitScenes, the real paired datasets it introduced alongside PU-Net and PC-Net. Uncertainty comes from repeated stochastic runs, with per-point variance taken over preserved indices. No reported result was found on Lacuna for P2P-Bridge's numeric scores on ScanNet++, ARKitScenes, PU-Net, or PC-Net, so we reproduce it ourselves under matched step counts. The deterministic manifold method Differentiable Manifold Reconstruction serves as an accuracy reference. It reports a point-to-surface error of 2.32 at 3% Gaussian noise, against 3.28 for the next-best learned baseline.
Three ablations isolate the key ingredients. The first replaces the pancake mixture with a single full-covariance Gaussian, the VRCNet-style unimodal choice. It tests whether a non-Gaussian family is needed. The second removes neighborhood-shared noise, giving a mean-field family over points. It tests whether coupled uncertainty matters in dropout regions. The third swaps forward KL for reverse-KL evidence-lower-bound training using the same likelihood and a learned surface-density prior. It tests the variance-underestimation hypothesis directly.
All methods share the same encoder capacity within 10% of parameters, the same training data, and the same hyperparameter-search budget. Every method, including ours, is also reported after split-conformal recalibration. That way we compare adaptive sharpness, not luck in calibration.
Why not just…?
Why not just predict a mean and a variance per point (heteroscedastic regression)? This is the simplest probabilistic denoiser, and it is our single-Gaussian ablation in its minimal form. Its mean is the conditional mean, which the shrinkage results in Score-Based Point Cloud Denoising and Differentiable Manifold Reconstruction predict will sit inside curved surfaces. Its variance mixes tangential slack, which is harmless, with normal error, which matters. We expect it to over-cover on flat regions and under-cover at edges. The curvature-stratified coverage plot will show whether this happens. If it matches us there, the mixture family is unnecessary and we will say so.
Why not just run Langevin sampling on an existing score model with several chains? This is the most serious threat. Each chain keeps point indices, so per-point sample variance is well defined, and with the right likelihood guidance it can approximate the true posterior asymptotically. We treat a guided annealed-Langevin sampler, built on a retrained score network, as the reference posterior for the amortization-gap study. We compare it on three things: coverage, interval width, and wall-clock time at matched quality. Two outcomes are possible. If Langevin with around 50 steps matches our calibration, our contribution reduces to amortized speed, and the note will be framed that way. If it is miscalibrated at practical step counts because of step-size sensitivity, that shows the value of the learned family.
Why not just use a deep ensemble of deterministic denoisers? Ensembles capture disagreement between models, not the width of the posterior given the data. On the synthetic oracle, where the posterior is known exactly, we can check whether ensemble spread tracks true posterior spread or collapses whenever members agree on a shared mode. The same test applies to flagging: if ensembles achieve comparable area under the curve for identifying points with error above tolerance, the variational machinery is not earning its cost.
Proposed Experiments
Synthetic check. The first decisive check uses PU-Net meshes at 1–3% Gaussian noise and the PointCleanNet test shapes, where the oracle posterior is exact given the true surface. We report the Wasserstein distance between $q$ and the oracle, normal-interval coverage at 50/90/95% stratified by curvature, and Chamfer and point-to-mesh error of the mode estimate. Success means Chamfer no worse than the reproduced score-based baseline at 3% noise, with coverage within two points of nominal in every curvature bin. The single-Gaussian ablation should miss nominal coverage at high curvature.
Real-data check. The second check uses ScanNet++ and ARKitScenes, which pair noisy consumer-sensor scans with laser ground truth. Coverage is measured against signed distance to the reference scan, stratified by range and local point density. We also report area under the curve for flagging points whose error exceeds 1 cm, and interval width at matched coverage. Registration error in the reference scans sets a floor on measurable coverage. We estimate that floor from overlapping reference scans and report it.
Transfer check. The third check tests out-of-distribution noise on Paris-rue-Madame outdoor LiDAR, where no clean reference exists. We run qualitative flagging analysis and check that intervals widen with range, which physics predicts. We report latency per 10,000 points for every method.
Scope. Completion of missing regions is excluded, because there are no observations to anchor the posterior. This is a stated limitation of the approach, not an oversight.
References
- FoldDiff: Folding in Point Cloud Diffusion
- Score-Based Point Cloud Denoising
- Differentiable Manifold Reconstruction for Point Cloud Denoising
- Rethinking Lossy Compression: The Rate-Distortion-Perception Tradeoff
- Diffusion Bridges for 3D Point Cloud Denoising
- Variational Relational Point Completion Network
- LION: Latent Point Diffusion Models for 3D Shape Generation
- Semi-Implicit Graph Variational Auto-Encoders
- Stochastic Solutions for Linear Inverse Problems using the Prior Implicit in a Denoiser
- Traversing Distortion-Perception Tradeoff using a Single Score-Based Generative Model