Geometric Refinement and Distributional Alignment for Sim2Real Roadside LiDAR Detection

Aman UpganlawararXiv 2026 · arXiv

TL;DR

In the UrbanTwin Sim2Real LiDAR Challenge, Aman Upganlawar demonstrates that massive performance gains in 3D object detection don't always require new architectures. By treating small-object failure as a localization problem and analytically "solving" realism metrics, this approach achieved a top-tier score (0.4793) through rigorous measurement discipline and surgical post-processing refinements.

What Problem Does This Paper Address?

The paper tackles the Sim2Real gap in roadside LiDAR detection. In this specific challenge (the LUMPI track), participants must train a 3D object detector exclusively on synthetic data but are evaluated on unseen real-world roadside LiDAR data.

The challenge is twofold:

  1. Detection Accuracy: Achieving high mean Average Precision (mAP) on real data despite training on a "digital twin."
  2. Distributional Realism: Ensuring the synthetic point clouds generated by the participant are statistically similar to real-world point clouds across metrics like Chamfer Distance and Frechet Point cloud Distance (FPD).

Roadside LiDAR presents unique challenges compared to vehicle-mounted sensors, such as fixed viewing angles and significant occlusion, which often lead to poor performance on smaller objects like pedestrians.

Methods

Instead of designing a new neural network, the author used a standard PointPillars baseline from the OpenPCDet toolbox. The real innovation lies in the "measurement discipline" applied to the pipeline:

  • Geometric Refinement: For small objects (Person class), the author replaced the predicted box center with the centroid of the LiDAR points inside it. This addresses quantization errors caused by the bird's-eye view (BEV) resolution.
  • Density-Based Re-ranking: Detections were rescored based on the number of interior LiDAR points, demoting sparse, low-confidence "ghost" boxes.
  • Analytical Realism Alignment: The author discovered that the Frechet Point cloud Distance (FPD) metric is a closed-form function of just nine summary statistics (mean and covariance of xyz coordinates). This allowed for a simple per-axis affine correction to "align" synthetic distributions to real ones without needing complex generative models.
  • Multiplexed Submissions: To maximize the limited submission budget, the author treated each class as an independent experiment within a single submission, allowing for rapid, controlled testing of multiple hypotheses.

Datasets

The paper utilizes the LUMPI (Leibniz University Multi-Perspective Intersection) dataset.

  • Synthetic Data: A digital twin sequence of a real intersection in Hannover, Germany.
  • Real Data: A public split of real LiDAR from the same site for calibration, and a hidden set of 50 frames for final evaluation.
  • Augmentation: The author synthesized additional "vulnerable road users" (pedestrians, cyclists) using empirical point-count curves and sensor models to fix under-representation in the original twin sequence.

Key Insights and Ideas

  • Localization vs. Detection: A key "aha moment" was realizing that pedestrians were being found by the detector but rejected by the metric because their centers were slightly off. Because pedestrians are small, a tiny physical error results in a massive drop in Intersection over Union (IoU).
Counterfactual analysis of Person AP errors
  • The "Silent Killer" Rule: An audit revealed a post-processing rule intended to help that was actually deleting valid Van predictions. Fixing this single bug resulted in a 335% relative gain for the Van class.
  • Metrics as White Boxes: By reading the scoring code, the author realized the realism metrics weren't a black box. The FPD metric could be substantially improved by simply matching the standard deviation of the synthetic data to the real reference.
  • Calibration over Simulation: The author argues that for Sim2Real, measuring the systematic biases (like object dimensions) between the two domains and applying simple offsets is often more effective than trying to build a "perfect" simulator.
Measured offsets between synthetic and real domains

Key Results

The final approach achieved a combined score of 0.4793, comprising a detection mAP of 0.1333 and a realism score of 0.9126.

  • Person AP: Improved by 86% relative (from 0.0346 to 0.0645) via geometric refinement.
  • Van AP: Improved by 335% relative on the evaluation server after fixing post-processing bugs.
  • Realism: The affine correction significantly boosted FPD and Earth Mover's Distance (EMD) scores.
Final quantitative results

Limitations and Future Work

The author identifies several "hard ceilings" in the current challenge:

  • The Bus Class: The ground truth for buses was found to be inconsistent (sometimes car-sized, sometimes two-wheeler sized), making it "unreachable" and costing participants roughly 1/7th of their potential mAP.
  • Realism Cap: Realism is capped at ~0.91 due to the underlying geometry of the digital twin; further gains would require regenerating the twin itself rather than post-processing the points.
  • Negative Results: Several complex techniques, like learned refiners and higher voxel resolutions, failed to provide gains, highlighting the importance of reporting what doesn't work.
Summary of negative results and failed levers

Summary updated

Abstract

We describe our approach to the LUMPI track of the UrbanTwin Sim2Real LiDAR Challenge, in which a 3D object detector must be trained exclusively on synthetic data and evaluated on unseen real roadside LiDAR, while the synthetic point clouds are simultaneously scored for distributional realism. Our final entry scores 0.4793 combined, from a detection mAP of 0.1333 and a realism score of 0.9126. Our contribution is not a new architecture. Working from a PointPillars baseline, we obtained most of our gains from three sources. First, a diagnosis that small object failure in roadside LiDAR is a localisation problem rather than a detection problem, which led to a parameter free geometric refinement worth 86 percent relative on pedestrian AP. Second, an audit of our own post processing which revealed that a shipped rule was destroying valid predictions, the repair of which produced our single largest detection gain at 335 percent relative on the Van class. Third, treating the realism metrics analytically rather than as a black box, which showed that one of the four components is a closed form function of nine summary statistics. We also report an experimental methodology that we believe is the more transferable result. Because mean AP decomposes into independent per class terms, a single submission can carry several mutually controlled experiments, and a submission assembled entirely from previously measured components has a score that is known before it is sent. We predicted our final score to four decimal places prior to submission. We report negative results at equal length, and give a full account of data provenance and compliance in Section 8, including a full account of every mechanism contributing to the submitted score.
BibTeX
@misc{upganlawar2026geometric,
  author = {Aman Upganlawar},
  title = {{Geometric Refinement and Distributional Alignment for Sim2Real Roadside LiDAR Detection}},
  year = {2026},
  eprint = {2611.03936},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.03936},
}
Figure