Empirical Comparison of Full-Waveform Lidar Algorithms

Christopher Parrish, Inseong Jeong, Robert D. Nowak, R. B. SmithPhotogrammetric 2011 · DOI

TL;DR

This paper introduces a novel empirical method for comparing full-waveform (FW) lidar processing algorithms using a controlled lab setup with precisely measured targets. They found that no single algorithm is universally superior, with EM deconvolution offering the best resolution under ideal conditions, while a hybrid approach provides a good balance of performance and robustness, and Gaussian decomposition is the fastest.

What Problem Does This Paper Address?

The increasing adoption of full-waveform (FW) lidar technology by commercial manufacturers has created a critical need for understanding and comparing the performance of various waveform processing algorithms. Unlike traditional discrete-return lidar, which captures only a few discrete range measurements per laser pulse, FW lidar records the entire backscattered waveform. This allows for more detailed analysis and potentially higher resolution, but it also necessitates sophisticated software-based processing. Software developers face the challenge of choosing which algorithms to implement, and users need to understand the trade-offs between different approaches in terms of accuracy, speed, and robustness for diverse applications like mapping, forestry, and obstruction surveying. This paper aims to provide a practical, empirical comparison of leading FW processing algorithms to guide these decisions.

Methods

The authors developed a controlled laboratory experiment to rigorously compare three distinct FW lidar waveform processing algorithms:

  • Gaussian Decomposition: This widely used method models the lidar waveform as a sum of Gaussian components, each representing a distinct laser reflection. The algorithm fits these components to the observed waveform.
  • EM Deconvolution: This approach attempts to recover the "true" undegraded signal from the observed, blurred, and noisy waveform by solving an inverse problem. It assumes the underlying signal is sparse (composed of a few localized spikes).
  • Hybrid (Wiener Filter + Gaussian Decomposition): This method combines Wiener filtering to reduce noise and blur with subsequent Gaussian decomposition to extract target information.

The core of their methodology involved a custom-built ranging lab with adjustable screen targets. The precise, measurable separations between these targets allowed for systematic evaluation of target resolution and separation accuracy as the distances were incrementally reduced. They also collected discrete-return lidar data simultaneously to quantify the improvements offered by FW processing.

Datasets

The study utilized data collected in a controlled laboratory environment. The "dataset" consists of waveforms recorded from an Optech ALTM Gemini lidar system (1064 nm wavelength, 167 kHz max PRF) and an Optech waveform digitizer. The experiment involved four targets: three adjustable screen targets and a fixed back wall target. The key innovation here is not a pre-existing benchmark dataset, but rather the creation of a controlled experimental setup that generates precisely characterized data for evaluating algorithm performance under varying target separations.

Key Insights and Ideas

  • No "One-Size-Fits-All" Solution: The most significant takeaway is that different algorithms excel under different conditions. The choice of algorithm and its parameters is highly application-dependent.
  • FW vs. Discrete-Return: The study empirically confirms the substantial improvement in target resolution offered by FW lidar processing compared to traditional discrete-return systems.
  • Robustness Matters: The performance of algorithms can be significantly impacted by factors like system stabilization and noise levels. EM deconvolution, while powerful, is sensitive to these variations, whereas the hybrid and Gaussian decomposition methods show more robustness.
  • Trade-offs are Inherent: Users must balance factors like target resolution, separation accuracy, processing speed, and robustness against parameter tuning.

Key Results

  • Target Resolution: All three FW algorithms significantly outperformed discrete-return lidar in resolving closely spaced targets. EM deconvolution achieved the best target resolution (115 cm at 70 kHz) under optimal conditions, followed by the hybrid approach (160 cm at 70 kHz) and Gaussian decomposition (205 cm at 70 kHz).
  • Target Separation Accuracy: EM deconvolution also showed the best bias-removed target separation accuracy (0.021 m at 70 kHz), indicating it could more precisely determine the distances between targets.
  • Processing Speed: Gaussian decomposition was the fastest algorithm, completing the processing in approximately half the time of the other two methods.
  • Impact of Noise/Stabilization: The 50 kHz data, collected before the system fully stabilized, showed increased noise and variability. This negatively impacted EM deconvolution's performance, highlighting its sensitivity to these factors, while the hybrid approach performed better in this scenario.

Limitations and Future Work

  • Controlled Environment: The study was conducted in a laboratory setting. While this allows for precise control, it doesn't fully replicate the complexities of real-world airborne data (e.g., varying atmospheric conditions, complex natural targets).
  • Limited Target Types: The use of screen targets, while effective for controlled separation measurements, may not fully represent the diverse reflectivity and geometric properties of natural surfaces.
  • Parameter Tuning Subjectivity: While efforts were made to optimize parameters, some subjectivity remains in tuning algorithms for peak performance.

The authors suggest future work could include:

  • Fusion-Based Approaches: Combining the results of multiple processing strategies might offer enhanced performance.
  • Complex Target Analysis: Extending the lab experiment with more diverse targets and configurations.
  • Airborne Data Validation: Applying and validating these methods on actual airborne lidar datasets, acknowledging the challenges of uncontrolled environments and reference data uncertainty.
  • Algorithm Modifications: Exploring modifications to algorithms, such as compressed sensing for EM deconvolution, to improve robustness and handle more complex scenarios.

Summary updated

Abstract

As third-party lidar software manufacturers are increasingly adding support for full-waveform data, a common question is which algorithm(s) to implement. To this end, a new approach is needed to compare and contrast various lidar waveform processing strategies from a practical, operational perspective. Quality and type of information output, processing speed, suitability for particular applications, robustness against poor parameter selection, and more subjective measures related to user experience are of interest. This paper describes a new empirical method of comparing range extraction and discrimination performance of different algorithms, based on a ranging-lab setup with multiple, adjustable screen targets, with precisely-measured separations. We present the results of comparing three different algorithms described in the scientific literature. The results show distinct differences and also indicate that there is no “one-size-fits-all” approach: the choice of a specific algorithm and adjustable parameter settings are highly application-dependent.
BibTeX
@inproceedings{parrish2011empirical,
  author = {Christopher Parrish and Inseong Jeong and Robert D. Nowak and R. B. Smith},
  title = {{Empirical Comparison of Full-Waveform Lidar Algorithms}},
  booktitle = {Photogrammetric},
  year = {2011},
  doi = {10.14358/pers.77.8.825},
}
Figure