LiDAR-in-the-Loop Hyperparameter Optimization

Félix Goudreault, Dominik Scheuble, Mario Bijelic, Nicolas Robidoux, Felix HeideCVPR 2023 · DOI

Abstract

In autonomous driving, LiDAR sensors are typically treated as fixed "black box" components that provide point clouds for downstream perception tasks. However, the internal Digital Signal Processing (DSP) that converts raw light waveforms into points relies on numerous hyperparameters often tuned manually for visual clarity rather than machine performance. This paper introduces a method to optimize these internal settings directly for tasks like 3D object detection. By utilizing a realistic transient simulator and derivative-free optimization, we achieve a 39.5% improvement in mean Average Precision (mAP) over standard expert-tuned configurations.

Introduction: Opening the Black Box

LiDAR is the cornerstone of depth perception for autonomous vehicles. Despite its importance, the hardware pipeline—specifically the DSP—remains largely unoptimized for the neural networks that consume its data. Parameters such as pulse power, detection thresholds, and peak selection logic are usually fixed by manufacturers to produce "clean" point clouds. Yet, what looks good to a human eye may discard critical information needed by an AI to detect a pedestrian in challenging conditions like fog or high ambient light.

Challenges in LiDAR Signal Processing

The conversion from raw light echoes to a discrete point cloud is fraught with noise and ambiguity. Standard DSPs must navigate several common pitfalls that can degrade the quality of the resulting point cloud for downstream AI tasks:

  1. Cluttered Surfaces: Multiple returns from complex geometries can create "ghost" points or obscure targets.
  2. Strong Retroreflectors: Highly reflective surfaces (like traffic signs) can saturate sensors or create side-lobe artifacts.
  3. Ambient Light: Solar radiation introduces a noise floor that can mask weak returns from distant objects.
Common challenges in LiDAR signal processing including cluttered surfaces, retroreflectors, and ambient light interference.

Methodology: Optimization in the Loop

To optimize these non-differentiable hardware settings, the authors developed a two-pronged approach centered on a "LiDAR-in-the-loop" architecture.

Transient LiDAR Simulation

The framework utilizes a physics-based forward model integrated with the CARLA simulator. Unlike traditional simulators that merely project 3D points, this model simulates raw temporal wavefronts. It accounts for pulse shapes, object reflectance via the Cook-Torrance model, and multipath transients. This high-fidelity simulation allows the optimizer to "see" how changing a hardware threshold affects the resulting waveform and, subsequently, the point cloud.

Derivative-Free Optimization

Because hardware DSPs are non-differentiable, standard backpropagation cannot be used. Instead, the authors employ a modified Covariance Matrix Adaptation Evolution Strategy (CMA-ES). This 0th-order stochastic algorithm iteratively refines sensor parameters to maximize a downstream metric, such as object detection accuracy, while balancing multiple goals using a Stable Dynamic Max-Rank Loss.

The proposed optimization pipeline, integrating transient simulation, waveform generation, and DSP logic into a closed-loop system.

Key Results

The "in-the-loop" shift yields significant performance gains across both simulated and real-world environments:

  • Detection Accuracy: For 3D object detection, the optimized settings outperformed manual expert tuning by 39.5% mAP.
  • Long-Range Perception: The most dramatic improvements occurred at ranges of 50–80 meters. In these scenarios, signals are typically weak; the optimizer learned to adjust thresholds to recover objects that were previously invisible to the "clean" expert-tuned settings.
  • Real-World Validation: Testing on a physical Baraja Spectrum LiDAR unit confirmed the simulation's accuracy, reducing point cloud histogram errors from 10.1 cm to 3.0 cm.

Conclusion and Future Work

This research demonstrates that the LiDAR sensor should be viewed as a programmable, task-dependent part of the AI stack. By treating sensor settings as hyperparameters, we can significantly enhance the reliability of autonomous perception. Future work aims to implement scene-dependent optimization, allowing vehicles to dynamically reconfigure their sensors in real-time to combat adverse weather or high-speed environments.

Summary updated

BibTeX
@inproceedings{goudreault2023lidar,
  author = {Félix Goudreault and Dominik Scheuble and Mario Bijelic and Nicolas Robidoux and Felix Heide},
  title = {{LiDAR-in-the-Loop Hyperparameter Optimization}},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year = {2023},
  doi = {10.1109/cvpr52729.2023.01288},
}
Figure