When Can Local Geometry Bridge Sim-to-Real? A Matching-Threshold Test for LiDAR Adaptation

Several methods that adapt a LiDAR segmentation model from synthetic to real scans work by matching small local neighborhoods of points across the two domains. They assume that a neighborhood in a dense, clean synthetic scan and its counterpart in a sparse, dropout-ridden real scan share enough structure to be put into correspondence. Graph-alignment theory studies exactly this question: when two noisy copies of a network can be lined up node for node, and when they cannot. It has a sharp answer for sparse random graphs. If each node keeps on average $\lambda s$ neighbors present in both copies, where $\lambda$ is the average number of neighbors and $s$ is the fraction of edges that survive in the second copy, then for $\lambda s \le 1$ no algorithm can recover even a small fraction of the true matches. This proposal asks whether a version of that threshold, measured directly on real sensor data, predicts where structure-based sim-to-real alignment helps and where it adds noise. It then asks whether gating the alignment by that prediction beats the best existing LiDAR adaptation methods.

The opportunity is practical. The sim-to-real literature reports large, persistent gaps. On SynLiDAR→SemanticKITTI the best adapted model in one study reaches 22.8% mean intersection-over-union (mIoU), against 55.5% for a model trained on real labels. The same literature also says plainly that it has no criterion for when transfer is feasible. Graph-alignment theory offers such criteria, but only for idealized models with symmetric noise and full node overlap. Neither field has tested its assumptions against the other's data.

Core Claim

Real LiDAR degrades synthetic geometry asymmetrically. The synthetic scan is "integral": every surface is captured. The real scan is a censored subset of it, thinned by range, beam pattern, absorption and occlusion. We claim three things. First, recovery of local point correspondences between a dense scan and its censored version shows a sharp transition, rather than a gradual decline, as a measured structure budget crosses a critical value. Second, that budget is the expected number of neighborhood edges surviving censoring, plus a contribution from per-point features. It predicts the transition better than raw point density does. Third, applying a local graph-matching alignment loss only in regions above the threshold, and falling back to self-training elsewhere, improves real-domain mIoU over both ungated graph matching and the strongest mixing baseline.

The first claim is an empirical test of transferring a theoretical idea, not a theorem. Point-cloud neighborhood graphs are geometric and full of triangles, so the tree approximation behind the sparse-graph result does not strictly hold. We treat the theory as a source of a falsifiable prediction about the shape and approximate location of a transition.

Grounding in Prior Work

The motivating evidence comes from LiDAR adaptation. SynLiDAR built a 198,000-scan Unreal Engine dataset. It factored the gap into appearance and sparsity, with a translator that handles sparsity on range images. That supports our decision to treat sparsity, or censoring, as the main mechanism. Adversarially Masking Synthetic to Mimic Real showed that domain discriminators exploit the "clean vs. missing points" shortcut, and it learns dropout masks to inject into synthetic scans. We reuse those masks as one realistic censoring model. Its success also raises a question it cannot answer: how much masking destroys the local structure that later alignment relies on? Unsupervised Domain Adaptation for Point Cloud Semantic Segmentation via Graph Matching is the closest method. It builds k-nearest-neighbor feature graphs around sampled centroids, matches source and target graphs with Sinkhorn optimal transport, and adds a local alignment loss. It applies that loss everywhere and never asks whether a given neighborhood holds enough shared structure to be matchable. Our proposal is a feasibility test and a gate for exactly this kind of loss. CoSMix swaps semantic patches between domains under teacher pseudo-labels. It is the strongest heuristic competitor and supplies our fallback branch.

The method-supplying theory comes from graph alignment. Impossibility of Partial Recovery in the Graph Alignment Problem gives the $\lambda s \le 1$ impossibility for sparse correlated random graphs. Aligning random graphs with a sub-tree similarity message-passing algorithm gives an algorithm that succeeds above it, and we use it as one matcher. Asymmetric tree correlation testing for graph alignment handles graphs with different densities, which is the right starting point for a dense synthetic scan against a sparse real one. Attributed Network Alignment shows that edge information and node-feature information add up toward the recovery threshold. That paper also flags two limitations: symmetric Gaussian noise and full node overlap. Both are violated by LiDAR, and our experiments probe them. Perfect Recovery for Random Geometric Graph Matching with Shallow Graph Neural Networks shows that one or two rounds of neighbor averaging let geometric graphs be matched under feature noise that defeats feature-only matching. That result supplies our graph-neural-network matcher, and it predicts that the feature term in the budget should be amplified by neighborhood aggregation.

Information-Theoretic Analysis of Unsupervised Domain Adaptation bounds target error through divergences and weight–data mutual information for i.i.d. vector data. It is a constraint on our framing rather than a competitor. It explains why global alignment can fail, but says nothing about relational structure. We do not claim to extend its bounds.

Research Program

The first component is a paired testbed with known correspondences. Sim-to-real benchmarks have no ground-truth point correspondence, so the theory cannot be tested on them directly. We build three censoring channels, each of which turns a dense scan into a censored one while keeping the identity of surviving points. The first removes beams from real SemanticKITTI 64-beam scans to mimic 32-beam and 16-beam sensors; this is real hardware censoring with exact correspondence. The second applies the learned dropout masks from the adversarial masking work to SynLiDAR scans. The third uses an empirical range-and-incidence-angle dropout model fitted to real SemanticKITTI return statistics and applied to SynLiDAR. In every case we partition each scan into local regions by range bin and semantic class, and build k-nearest-neighbor graphs in the dense and censored versions.

The second component is a structure budget computed for each region. For each region we estimate $\lambda$, the mean neighborhood degree in the dense scan, and $s$, the fraction of those neighborhood edges that are still edges among surviving points after censoring. We record the product $\lambda s$, the expected number of surviving shared neighbors per point. The sparse-graph theory predicts recovery fails below

$$\lambda s = 1 .$$

To this we add a feature term $\phi$: the empirical correlation, between the dense and censored scans, of a frozen backbone's per-point embeddings at corresponding points. Following the additive principle in the attributed-alignment work, we fit the threshold as a line in the $(\lambda s, \phi)$ plane on one channel. We then test whether the same line transfers to the other two. Because geometric graphs violate the tree assumptions, we expect the critical value of $\lambda s$ to shift away from 1. The prediction under test is that a single transition curve exists and transfers across channels.

The third component is a set of matchers and a phase diagram. In each region we run three algorithms that recover the correspondence and score overlap, the fraction of points correctly matched. They are sub-tree message passing, Sinkhorn optimal transport on backbone features as in the graph-matching adaptation method, and a two-layer graph neural network followed by Hungarian assignment. Plotting overlap against the budget gives the phase diagram. The gap between the best matcher's transition and where the budget says information runs out is itself informative: it shows how much of the LiDAR gap is algorithmic rather than informational.

The fourth component is budget-gated adaptation. On the actual unpaired benchmarks we cannot compute $s$ from correspondences. We can, however, estimate it per range bin and class from the paired channel that best matches the target sensor, since $\lambda$ comes from synthetic scans and the censoring statistics come from real sensor models. We then apply the local graph-matching alignment loss only to centroids whose region lies above the fitted transition. Below it we use CoSMix-style mixing and self-training. The point is not a new loss. The point is that misapplied alignment in information-poor regions, such as distant small objects under 32 beams, actively pulls features toward wrong matches.

Baselines

The naive reference is the source-only model trained on SynLiDAR and tested on the real target. Its reported mIoU is 15.2% on SemanticKITTI and 11.5% on nuScenes, in Adversarially Masking Synthetic to Mimic Real. Any method must clear it substantially, and we rerun it with our backbone to anchor comparisons.

The strongest existing method on SynLiDAR→SemanticKITTI is CoSMix, which reports 32.2 mIoU in its own paper. The adversarial masking paper reports CoSMix at 20.0% and itself at 22.8%, presumably under a different protocol or class mapping. Because of this discrepancy we will rerun CoSMix and the masking method from released code, under one class mapping and one backbone, with matched training iterations and an equal hyperparameter search budget for every method. On SemanticKITTI→nuScenes the strongest reported number is the graph-matching method's 37.3% mIoU, against 31.6% for the prior best, in Unsupervised Domain Adaptation for Point Cloud Semantic Segmentation via Graph Matching. We will reimplement it, since it is also our ungated ablation. Beating these means a gain in mean mIoU over our own reruns, averaged over three seeds, that exceeds seed variance. We expect the gain to be concentrated in distant and small classes, where the budget predicts alignment is infeasible.

The key ablations isolate the gate. Ungated graph matching applies the same loss everywhere. A density gate uses the same coverage fraction but thresholds raw surviving point count. A random gate also keeps coverage matched. A structure-only gate drops the feature term $\phi$. If the density gate matches ours, the graph-theoretic budget adds nothing and we will say so.

For the phase-diagram study, the baseline predictor of matching overlap is surviving point density alone. Our budget must give higher held-out explained variance and a sharper, channel-transferable transition. No reported result was found on Lacuna for correspondence-recovery overlap on LiDAR scans, so these numbers are new measurements.

Why not just…?

Why not just gate by point density? This is the most serious threat. Density and $\lambda s$ are correlated, and if density predicts matchability equally well, the theory is decoration. The two can come apart: a dense region of flat road has many points but neighborhoods that look alike everywhere, a symmetry the theory says destroys identifiability, while a moderately sparse pole keeps distinctive structure. The density-gate ablation and the held-out phase-diagram comparison settle this directly. We commit to reporting a negative result if density wins.

Why not just gate by pseudo-label confidence, as self-training already does? CoSMix-style methods filter by teacher confidence. Confidence is miscalibrated exactly under severe shift; the research file notes that pseudo-labels degrade under 64→32 beam reduction. The budget is computed from geometry and sensor statistics, independent of the classifier. We include a confidence-gated variant with matched coverage. The budget should win specifically on the nuScenes target, and if it does not, the claim fails.

Why not just tune the masking strength of adversarial masking or add more augmentation? Better augmentation shrinks the gap but does not tell you where structure is unrecoverable, and over-masking can push synthetic regions below the threshold. Our paired study measures this: we sweep the mask rate and check whether downstream mIoU peaks where the budget of masked synthetic regions crosses the transition. A peak at that point would turn the theory into a tuning rule. If masking strength alone closes the gap to our gated method, the gating contribution is moot.

Why not just align global feature distributions? The research file already shows global alignment underperforms. DANN reaches 57.7% against 71.9% for KL with gradient penalty on VisDA17 in Information-Theoretic Analysis of Unsupervised Domain Adaptation. Global alignment also blurs pole and tree categories, and discriminators exploit dropout shortcuts. We include a global-alignment rerun only as a reference.

Decisive Checks

The work stands or falls on four observations, all on SynLiDAR, SemanticKITTI and nuScenes. First, on the beam-removal channel, matching overlap must drop sharply over a narrow band of the budget for all three matchers. The transition curve fitted on beam removal must predict the transition on the masking and dropout-model channels better than density does. Second, the gap between the message-passing and optimal-transport transitions tells us whether existing Sinkhorn-based alignment is information-limited or algorithm-limited. Third, per-class adaptation gain from ungated graph matching should be negative or near zero for classes whose regions sit mostly below the transition, such as distant persons and bicycles under 32 beams. This should hold with a rank correlation between predicted budget and per-class gain that clearly exceeds the density predictor's. Fourth, the gated method should beat our reruns of CoSMix and ungated graph matching on SynLiDAR→SemanticKITTI and SynLiDAR→nuScenes, with gains concentrated where the budget says they should be. Failure of the first check ends the theoretical claim. Failure of only the last two leaves a useful diagnostic but no method.