Abstract
Lidar sensors are costly yet critical for understanding the 3D environment in autonomous driving. High-resolution sensors provide more details about the surroundings because they contain more vertical beams, but they come at a much higher cost, limiting their inclusion in autonomous vehicles. Upsampling lidar pointclouds is a promising approach to gain the benefits of high resolution while maintaining an affordable cost. Although there exist many pointcloud upsampling frameworks, a consistent comparison of these works against each other on the same dataset using unified metrics is still missing. In the first part of this paper, we propose to benchmark existing methods on the Kitti dataset. In the second part, we introduce a novel lidar upsampling model, HALS: Height-Aware Lidar Super-resolution. HALS exploits the observation that lidar scans exhibit a height-aware range distribution and adopts a generator architecture with multiple upsampling branches of different receptive fields. HALS regresses polar coordinates instead of spherical coordinates and uses a surface-normal loss. Extensive experiments show that HALS achieves state-of-the-art performance on 3 real-world lidar datasets.
Introduction
High-resolution Lidar is essential for safe autonomous driving, providing the dense 3D data needed for precise object detection and scene understanding. However, the cost of high-beam sensors (e.g., 64 or 128 beams) remains a significant barrier to mass-market adoption. Lidar super-resolution offers a software-based solution to upsample data from affordable low-resolution sensors. This paper addresses two critical gaps in the field: the lack of standardized benchmarks on real-world data and the failure of existing models to account for the unique vertical geometry of Lidar scans.
Methodology: Height-Aware Super-Resolution
The core innovation of HALS (Height-Aware Lidar Super-resolution) stems from the observation that Lidar range distributions are inherently height-dependent. In a range image, upper rows typically capture distant objects (higher mean range) with greater variance, while lower rows capture the nearby ground plane.
To exploit this, HALS utilizes a grid-based architecture with several key features:
- Multi-Scale Receptive Fields: The generator employs parallel branches. A "shallow" branch with a small receptive field processes the upper scan regions (small, distant objects), while a "deep" branch with a larger receptive field handles lower regions (large, near objects).
- Confidence-Based Fusion: The model learns to blend these branches using predicted confidence maps, ensuring optimal feature integration.
- Polar Coordinate Regression: By regressing radial distance ($d$) and height ($z$) instead of standard spherical coordinates, HALS minimizes vertical quantization errors common in real-world sensors.
- Virtual Normal Loss: A surface-normal loss is incorporated to preserve sharp geometric edges and prevent the "blurring" effect often seen in CNN-based upsampling.
Key Results
HALS was evaluated against state-of-the-art methods on KITTI Raw, KITTI Object, and NuScenes datasets. It consistently outperformed competitors in both geometric accuracy and downstream task performance.
Geometric Accuracy
On the KITTI Raw dataset (4x upsampling), HALS achieved an Earth Mover’s Distance (EMD) of 82.0 and a Chamfer Distance (CD) of 0.015, significantly better than the previous leading grid-based model, ILN (EMD: 104, CD: 0.061). Qualitative results (see Figure 1) show that HALS maintains the structural integrity of objects like cars and flat surfaces like roads, avoiding the noisy artifacts produced by other methods.
Downstream Impact
The utility of HALS extends to perception tasks. When using upsampled data for car detection with a PointPillars detector on the KITTI Object dataset, HALS achieved a 55.76% Average Precision (AP) in the "Easy" category, outperforming ILN's 51.93%.
Discussion and Insights
A key finding of this work is that grid-based methods (2D CNNs) are superior to point-based methods (e.g., PointNet) for Lidar upsampling. 2D kernels effectively bridge the gap between vertical scan lines, which is crucial for filling in missing beams. Furthermore, the height-aware approach directly addresses the non-uniform distribution of Lidar data, leading to more stable and accurate reconstructions across the entire field of view.
Conclusion and Future Work
HALS demonstrates that incorporating domain-specific geometric priors—specifically height-aware range distributions—is vital for high-quality Lidar super-resolution. While HALS sets a new state-of-the-art, future work could explore probabilistic generative models (like GANs or Diffusion) to better handle the inherent ambiguity in upsampling and optimize the framework for real-time deployment on embedded vehicle hardware.
Summary updated