A single LiDAR sweep sees only the side of an object that faces the sensor, and a distant pedestrian may return a few dozen points. Driving logs, however, record the same agent many times as it moves. Stacking those observations into one denser shape works well for cars. A car is rigid, so one rotation and translation per frame lines its points up. A walking person is not rigid. Their legs swing, their arms move, and their coat flaps. Stacking their sweeps with a single rigid transform produces a smeared, multi-limbed blur. This proposal builds an accumulator that first decides, for each tracked agent, how much non-rigidity its observations actually require, and then aligns the sweeps with the simplest motion model that explains them. The options are one rigid transform, a few rigid parts, or a smooth local deformation. The alignment uses a loss that ignores surfaces missing from a given sweep.
The payoff is a source of self-supervised, real-sensor completion targets for the agents that current temporal-accumulation methods exclude. Those targets can train a fast completion network that helps exactly where data is sparsest: small, deformable, distant road users. Two sentences capture it. Existing accumulation methods assume rigid objects and existing non-rigid registration methods assume isolated, scene-free shapes. We put a per-agent rigid/non-rigid decision in front of an occlusion-robust alignment, so pedestrians and cyclists can be densified from logs the way cars already are.
Core Claim
Temporal accumulation is currently built on a rigidity contract. Dynamic 3D Scene Analysis by Point Cloud Accumulation separates static background from moving objects and estimates one rigid transform per object "tube" across frames. Self-supervised Point Cloud Completion on Real Traffic Scenes via Scene-concerned Bottom-up Mechanism (TraPCC) pools a vehicle's sweeps into a pseudo-ground-truth pool and adds left-right symmetry. Both papers motivate this proposal, because both report gains from accumulation. Both also constrain it, because both are explicitly limited to rigid, symmetric vehicles.
The non-rigid side has the reverse limitation. Occlusion-aware Non-Rigid Point Cloud Registration via Unsupervised Neural Deformation Correntropy (OAR) shows that Chamfer distance tears or collapses shapes when the target is partial. It fixes this with the Maximum Correntropy Criterion and a Locally Linear Reconstruction prior. The Maximum Correntropy Criterion is a robust loss that gives near-zero weight to points with no counterpart. The Locally Linear Reconstruction prior lets unseen points follow their visible neighbours. OAR supplies our alignment loss. However, it runs on single isolated shapes and models everything as a free deformation field.
Our claim is that the missing ingredient is model selection over rigidity. The accumulator should fit progressively richer motion models and stop at the first one whose robust residual reaches sensor noise. A free deformation field fitted to a sparse, one-sided pedestrian scan has enough freedom to "explain" noise and occlusion holes. Constraining motion to be as rigid as the data allows is what keeps the accumulated shape physically plausible.
We use one display equation. For an agent tube with sweeps $P_1,\dots,P_T$, we choose the motion model $m$ from the ordered family rigid, $K$-part piecewise-rigid, and piecewise-rigid plus smooth residual deformation. The choice minimizes
$$ \mathcal{S}(m) = \sum_{t} \sum_{p \in P_t} \Big(1 - \exp\big(-d_m(p)^2 / 2\sigma^2\big)\Big) + \lambda \, k(m), $$
where $d_m(p)$ is the distance from point $p$ to the canonical shape after warping by model $m$, and $\sigma$ is the range-dependent LiDAR noise scale. The first term is the correntropy loss, so points on surfaces absent from the canonical shape contribute a bounded penalty instead of pulling the shape. $k(m)$ counts the model's motion parameters, and $\lambda$ is a complexity weight set once on validation data. Under this criterion, "rigid versus non-rigid" stops being a semantic label. It becomes a measured outcome per agent and per time window. A cyclist coasting may select rigid, and the same cyclist pedalling may select two or three parts.
Research Program
The pipeline starts where static-scene methods stop. Following the static/dynamic decomposition of Dynamic 3D Scene Analysis by Point Cloud Accumulation, background points are used to solve ego-motion. Points that violate the static-world assumption are clustered into agent tubes. We reuse this front end rather than reinvent it. The static scene assumption is therefore the regime we extend, not the contribution itself.
For each tube we build a canonical shape in two parts. The first is a set of skinning weights that assign each canonical point softly to $K$ rigid parts, with $K$ chosen from 1 to 4. The second is an optional small deformation field regularized by Locally Linear Reconstruction. Both parts are optimized under the correntropy objective above, with $\sigma$ tied to range. The selection rule then picks the motion model.
We add one more constraint, borrowed in spirit from EnerGS: Energy-Based Gaussian Splatting under Partial Geometric Priors. That paper penalizes geometry placed in space a sensor ray verified as empty, but assumes a static world. We apply the same free-space test per frame, in each agent's own warped frame. A canonical point that, warped to time $t$, lands in space the sweep at $t$ saw through is penalized. This makes free-space carving compatible with moving agents instead of corrupted by them, which is the failure EnerGS reports.
The accumulated canonical shapes become completion targets. Following TraPCC's pool idea but dropping its symmetry prior, which pedestrians break, we train a single-sweep completion network on real partial-sweep/accumulated-shape pairs. For speed we use the one-step average-velocity generator of 3D MeanFlow: One-Step Point Cloud Completion and Generation via Average-Velocity Transport. MeanFlow is a generator that maps noise to a completed point set in one network evaluation, reported at about 5 ms per shape. Its training targets have so far been synthetic or vehicle-only.
Baselines
The naive reference is the raw single sweep. Close behind it is naive ego-motion accumulation of dynamic points, which the reviewer-proposed Alternative 2 suggests and which Huang et al. show produces motion smear. Both run on identical tubes and splits.
The strongest existing accumulation method is TubeNet's single rigid transform per tube from Dynamic 3D Scene Analysis by Point Cloud Accumulation. On Waymo it reports dynamic-object error falling from 23.7 cm to 17.3 cm, and static background error falling from 12.9 cm to 1.8 cm. We run it with the same front end and the same tubes. Our method must match it on vehicles, where rigid should be selected, and beat it on pedestrians and cyclists.
The strongest non-rigid competitor is OAR applied per tube. OAR reports 89.7% relaxed accuracy on occluded TOSCA dogs, against 80.8% for DPF. We give OAR the same per-tube iteration budget and the same correntropy kernel width. This isolates the value of rigidity selection from the value of the robust loss.
RGGT, from RGGT: A Generative-Prior-Guided Transformer for Unified Rigid and Non-Rigid Point Cloud Registration, is the closest "unified rigid and non-rigid" method. It reports a 14.0-point gain in strict accuracy over GraphSCNet on 4DLoMatch; the file gives no absolute value. It is pairwise, so we chain it frame-to-frame within a tube. We note its roughly 10 s per pair feature-extraction cost as part of the comparison.
For the completion stage, the baselines are MeanFlow trained on synthetic data only, MeanFlow trained on TubeNet-accumulated targets, and TraPCC. TraPCC reports a mean Chamfer distance of 0.0973 on KITTI vehicles. The file reports no Lacuna number for TraPCC or MeanFlow on pedestrians or cyclists. All completion models share architecture, data volume and training steps; only the targets differ.
Three ablations isolate our key ingredient. The first forces a single fixed model for all agents: always rigid, always 3-part, or always deformation field. The second replaces correntropy with Chamfer distance. The third removes the per-frame free-space term.
Why not just…?
Why not just use a free deformation field for every agent, as OAR does? This is the most serious threat. If OAR-per-tube matches our held-out-sweep scores, the selection step adds nothing. Two considerations suggest selection will matter. OAR's reported results are on dense, isolated shapes, and real pedestrian sweeps are far sparser and more one-sided. With so few points, a free field has many unconstrained parameters per observed point. The forced-model ablation decides this directly, on the held-out-sweep and free-space metrics. If "always deformation" wins, we will report that, and the contribution reduces to applying robust non-rigid accumulation at scene scale.
Why not just use piecewise-rigid parts everywhere, or a parametric human body model? A fixed $K$-part model may be enough for pedestrians, and a body template is a strong prior. Both fail outside their assumptions. A fixed part count over-fits rigid agents. A human template does not cover cyclist-plus-bike, strollers, or animals, and it needs reliable class labels on 20-point clusters. The forced-$K$ ablation, broken down by class, shows whether adaptive selection beats the best fixed $K$ per class. If it only ties the best per-class fixed $K$, the honest conclusion is that class-conditioned fixed models suffice.
Why not just train completion on synthetic shapes and skip accumulation? Learning 3D Shape Completion From Laser Scan Data With Weak Supervision reports that a model fully supervised on synthetic data reaches 17.6 cm completeness error on KITTI, against 9.1 cm when exposed to real scans. TraPCC reports the same domain gap. Our synthetic-only MeanFlow baseline measures this gap for deformable classes specifically.
Why not just mask dynamics with a 2D tracker? Masking removes the agents instead of completing them, so it answers a different question. 3DGS-HPC and Split4D also document flicker and drift in 2D masks. Our front end uses only LiDAR geometry.
Proposed Experiments
The main evaluation runs on Waymo Open and nuScenes pedestrian and cyclist tubes. We also evaluate vehicle tubes as a control, where the selector should choose rigid and match TubeNet. Real data has no complete ground truth, so the main metric is held-out-sweep consistency. We accumulate a tube excluding sweep $t$, warp the canonical shape to time $t$, and measure two things. The first is one-sided distance from the observed points at $t$ to the prediction; this checks whether the prediction explains what was seen. The second is the fraction of predicted points lying in space the rays at $t$ passed through; this checks whether the prediction hallucinates into verified empty space. A method passes only if it improves both together, which blocks the trivial wins of inflating or shrinking shapes.
On real data we also report the distribution of selected models by class and speed. This gives an interpretable rigid/non-rigid map. We further report the downstream effect of the trained completion network on far-range pedestrian and cyclist detection. 3D MeanFlow reports roughly 45–50% relative mAP gains for cars beyond 50 m; the file has no comparable pedestrian number.
To check alignment against true correspondences, we test the accumulator on dense mesh sequences with synthetic LiDAR-style occlusion. The Articulated Mesh Animation sequences are one such set; DynoSurf reports a Chamfer distance of 0.320 there. The occluded TOSCA dog set is another, where OAR reports 89.7% relaxed accuracy. We also use 4DLoMatch at low overlap. On these, we sweep point density downward to locate the sparsity floor. That floor is the density at which every method's held-out error stops beating the single-sweep baseline.
The decisive result is narrow. On real pedestrian and cyclist tubes, adaptive selection should beat both TubeNet and OAR-per-tube on held-out-sweep distance and free-space violation together, while staying within noise of TubeNet on vehicles. Completion networks trained on its targets should then beat synthetic-only and TubeNet-target training on the same consistency metrics. If OAR-per-tube ties us, the rigidity-selection hypothesis is falsified for these classes.
References
- Dynamic 3D Scene Analysis by Point Cloud Accumulation
- Self-Supervised Point Cloud Completion on Real Traffic Scenes Via Scene-Concerned Bottom-Up Mechanism
- Occlusion-aware Non-Rigid Point Cloud Registration via Unsupervised Neural Deformation Correntropy
- EnerGS: Energy-Based Gaussian Splatting with Partial Geometric Priors
- 3D MeanFlow: One-Step Point Cloud Completion and Generation via Average-Velocity Transport
- RGGT: A Generative-Prior-Guided Transformer for Unified Rigid and Non-Rigid Point Cloud Registration
- Learning 3D Shape Completion From Laser Scan Data With Weak Supervision
- 3DGS-HPC: Distractor-free 3D Gaussian Splatting with Hybrid Patch-wise Classification
- Split4D: Decomposed 4D Scene Reconstruction Without Video Segmentation
- DynoSurf: Neural Deformation-Based Temporally Consistent Dynamic Surface Reconstruction