Geometry-Blind Discriminators for Cross-LiDAR 3D Detection

When a 3D object detector trained on one LiDAR sensor is moved to another, accuracy collapses. The shift can be from a 64-beam roof-mounted unit to a 16-beam unit on a knee-high delivery robot. One common repair is adversarial domain adaptation: a small classifier, the domain discriminator, tries to tell which sensor a feature came from, and the detector is trained to fool it. This makes features look the same across sensors. The repair has a hidden cost. Some of what differs between sensors is information the detector needs to keep. Cars in one country are genuinely smaller than in another, and a low-mounted sensor sees the ground at a different height. A box regressor has to report those differences, not erase them. If the discriminator can see object size and placement, fooling it means pushing target-domain box features toward source-domain sizes and heights. The predicted box then drifts toward the wrong answer.

We propose to split what the discriminator is allowed to see. The classification branch is aligned as usual, because "is this a car?" does not depend on sensor pose or object size. The box-regression branch is aligned only after its features have been re-expressed in each object's own predicted frame: centered, rotated to its predicted heading, and scaled by its predicted size. In that frame the discriminator cannot read off size, heading, or height. It can only detect what should be removed: point density, beam-ring patterns, and missing returns. The result is adversarial alignment that removes sensor signatures without flattening real geometry. It needs no target labels and no target size statistics.

Core Claim

Adversarial alignment applied to localization features is a learned invariance. It is harmful for the same reason data augmentation can be harmful. A Theory of PAC Learnability under Transformation Invariances shows that forcing invariance across a whole transformation orbit, when the target is invariant only on part of it, smears labels and can do strictly worse than plain training. Box targets are equivariant to translation, heading and scale, meaning they change predictably with them. Box targets are invariant only to the non-geometric sensor factors: density, beam layout and dropout.

The claim is that the right adversary for the box branch is one made blind to the equivariant factors by canonicalization. Here canonicalization means mapping each object into its own predicted reference frame, the alternative to building equivariance into layers studied in Affine Steerable Equivariant Layer for Canonicalization of Neural Networks. With this adversary, adaptation should close a specific, measurable error. Coupled adversarial methods should leave a target-domain size and height bias in predicted boxes, and the geometry-blind discriminator should remove most of it. This is the failure CL3D reports as unresolved: box edges and scale stay wrong even after class semantics transfer.

Relation to Prior Work

The closest work sets the question but does not answer it. Decoupled Adaptation for Cross-Domain Object Detection separates a category adaptor from a box adaptor in 2D detection. It motivates this proposal, since it shows that alignment on shared features hurts foreground discrimination. It differs in mechanism. D-adapt adapts boxes through an IoU-disparity discrepancy and iterated pseudo-label retraining, and it never confronts metric 3D geometry such as real object sizes or sensor height.

UADA3D supplies our base method. It uses a gradient reversal layer, which flips the sign of the discriminator's gradient so the detector learns to fool it. It adds class-wise discriminators and masks features by the predicted box, so the adversary looks only at objects. But its masked features still carry absolute size, heading and height, and it uses coupled heads. We keep its masking idea and change what the discriminator sees.

LiDARNet separates domain-private from domain-shared features with an orthogonality loss. Its split is learned, not geometrically defined, and it works on 2D range images for segmentation, not 3D boxes. ST3D and ST3D++ constrain the design in two ways. Their Random Object Scaling, and the statistical normalization variant, show that object-size mismatch is a first-order cause of cross-sensor loss. That normalization uses target size statistics, which we do not assume. Their reliance on self-training is reported by UADA3D to break on large sensor gaps.

Equivariant Point Network for 3D Point Cloud Analysis supplies an optional alternative mechanism. It factorizes 3D-rotation-equivariant convolution so that it becomes affordable. We use a heading-equivariant variant of it as one arm, to test whether equivariance by construction beats canonicalization for the box branch.

Research Program

The detector is CenterPoint, matching UADA3D's setup. Its shared backbone feeds two branches with separate parameters. One branch handles classification and center-heatmap scoring. The other regresses box offsets, size, height and heading, following the decoupling argument of Lu et al..

The classification branch receives a class-conditional discriminator on box-masked features, as in UADA3D. For the box branch, we gather the points and features inside each predicted box in both domains. Each point is mapped to object coordinates: subtract the predicted center, rotate by the predicted heading, and divide by the predicted length, width and height. We pool the canonicalized point features, so that density and beam structure survive while metric geometry is factored out. A second discriminator acts on this pooled descriptor, again through gradient reversal. Predicted boxes are used in both domains rather than ground truth on the source. Otherwise the discriminator could separate domains by box-quality noise alone.

The working hypothesis can be stated as an objective. Let $f_c$ and $f_b$ be classification-branch and box-branch features. Let $\pi_{\hat b}$ be the map into the frame of predicted box $\hat b$. Let $D_c, D_b$ be the two discriminators and $\lambda_c, \lambda_b$ their weights. The detector minimizes

$$ \mathcal{L}_{det}^{src} - \lambda_c \, \mathcal{L}_{adv}(D_c, f_c) - \lambda_b \, \mathcal{L}_{adv}(D_b, \pi_{\hat b} f_b), $$

while the discriminators minimize their own domain-classification losses $\mathcal{L}_{adv}$.

The analysis component asks what alignment destroys. On source and target, we fit linear probes from box-branch features to ground-truth size and height, and from discriminator inputs to the same quantities. A geometry-blind discriminator input should give near-chance probe accuracy for size, while box-branch features keep it. Under coupled UADA3D-style alignment, we expect probe-recoverable target size to shrink toward the source mean over training. That would be direct evidence of the label-smearing mechanism, not merely a downstream mAP difference.

Baselines

The naive reference point is source-only CenterPoint with standard global rotation, flip and scaling augmentation. UADA3D reports 15.96 mAP for it on Waymo→nuScenes.

The strongest competitor is UADA3D itself. It is the only end-to-end adversarial method shown to work on the large robot gap. It reports 26.89 mAP on Waymo→nuScenes and 31.54 mAP on nuScenes-32-beam→LAURA robot. We will re-run it with the same backbone, schedule, augmentation and discriminator capacity as our method, and with the same hyperparameter budget for its adversarial weight. A clear win requires two things. First, improving on our reproduced UADA3D mAP by more than its seed-to-seed spread, measured over three seeds. Second, showing that the gain comes from lower size, height and heading error on matched true positives rather than from more detections.

We will port D-adapt's decoupled category and IoU-disparity box adaptors onto the same 3D detector, with the same pseudo-label rounds. This tests whether the "obvious" 2D-to-3D transfer already suffices. ST3D++ is a self-training reference. On Waymo→KITTI, ST3D reports 61.83 AP3D for cars, and 73.37 with statistical normalization, which uses target size statistics. We report the latter separately as a privileged upper reference, not a fair competitor.

Two ablations isolate the key ingredient. The first, a decoupled-but-geometry-visible variant, gives the box discriminator box-masked features without canonicalization. The second, a detached variant, aligns only the classification branch and shields the box branch from adversarial gradients entirely.

Why not just…?

Why not just normalize the input clouds, by fitting a ground plane and resampling, and then run UADA3D? This is a real threat for sensor height and tilt, because those factors are global and estimable without labels. We include it as a preprocessing step for every method. Our claim is therefore restricted to what input normalization cannot fix: per-object size and heading. These have no global canonical value, and correcting them without labels is what statistical normalization needs target statistics for. If ground-plane normalization plus UADA3D matches us on size error, the claim is refuted.

Why not just keep the adversary away from the box branch? This is the detached ablation. If localization only needs to avoid harmful alignment, detaching should equal our method. Our hypothesis says the opposite: density and dropout signatures in box features also hurt regression, which is consistent with the missing-point account in SPG. So detaching should leave a residual error, concentrated on sparse, far-range objects. We will report errors binned by range and points-per-box to see where any gap lies.

Why not just use self-training with strong augmentation, as in ST3D++? On moderate gaps it may well win, and we will report that honestly on Waymo→KITTI. On the robot gap, UADA3D reports that ST3D++ and MS3D++ pseudo-labels collapse, which leaves adversarial methods as the viable family there.

Why not build heading equivariance into the layers instead of canonicalizing? We run an SPConv-style discrete heading-group arm for the box branch. Heading is a one-dimensional group, so the sample-complexity advantage of equivariance over augmentation argued by Shao et al. is likely small here. We expect canonicalization to match it at lower cost. If the equivariant arm wins, that would favor equivariance by construction even for small groups.

Proposed Experiments

We evaluate four adaptation pairs, all named in the file: Waymo→nuScenes, nuScenes-32-beam→LAURA robot (subject to data access via UADA3D), Waymo→KITTI, and nuScenes→PandaSet. The last is mechanical to solid-state, where CL3D reports 0.705 mAP and 77.03% of the gap closed. Metrics are mAP, AP3D, and BEV AP, the overhead-view variant that is less sensitive to height errors. We also report the true-positive error decomposition: center, size, height and heading error, together with the signed mean size bias of target predictions.

Three checks would settle the proposal. On Waymo→KITTI, where the car-size gap is known, signed size bias under our method should be substantially closer to zero than under UADA3D, without using target statistics. The geometry-visible ablation should reproduce UADA3D's bias. That would show canonicalization, not decoupling alone, is responsible. On the robot pair, our method should beat reproduced UADA3D and the detached ablation, with gains concentrated on sparse objects.

If none of these hold, and the gains are only in classification AP, the conclusion is that 3D localization under sensor shift is limited by sparsity rather than by alignment-induced geometry loss. That would point the field toward completion methods like SPG and away from smarter adversaries.