Diffusion anisotrope et inpainting d'orthophotographies LiDAR mobile

Pierre Biasutti, Jean–François Aujol, Mathieu Brédif, Aurélie BugeauHAL 2016

TL;DR

This paper presents a novel pipeline for generating high-resolution (1cm) reflectance and height orthophotos from mobile LiDAR scans. By combining anisotropic diffusion with a height-aware inpainting method, they effectively address the challenges of sparse data and occlusions inherent in LiDAR data, leading to more complete and accurate urban mapping.

What Problem Does This Paper Address?

Traditional methods for creating orthophotos and Digital Surface Models (DSMs) using satellite or aerial imagery often struggle to achieve very high precision (better than 5cm) and are prone to significant occlusions caused by objects like trees and buildings. Mobile LiDAR scanning offers a ground-level perspective that can overcome these limitations, providing dense 3D point clouds with high resolution. However, directly projecting these point clouds onto a 2D grid results in sparse, noisy images with missing data (holes) due to under-sampling and occlusions. This paper tackles the problem of interpolating these sparse point clouds and filling these occlusions to produce dense, accurate orthophotos of both reflectance and height.

Methods

The proposed pipeline consists of several key steps:

  1. Point Cloud Projection: The initial step involves projecting the 3D LiDAR point cloud onto a 2D pixel grid. This process includes filtering points based on height to remove ground points and objects that would otherwise create unwanted artifacts. The result is two sparse and noisy images: one for reflectance and one for height.
  2. Anisotropic Diffusion: To densify the sparse images and reduce noise, the authors employ a coupled anisotropic diffusion method. This technique allows for the simultaneous diffusion of both reflectance and height information. The diffusion process is guided by the gradients of both images, ensuring that diffusion is stronger in homogeneous regions and weaker at edges, thus preserving important structural details.
  3. Denoising: Following diffusion, the images may still contain artifacts, particularly "salt-and-pepper" noise resulting from oversampling in certain areas. This noise is reduced using a TV-L¹ regularization-based denoising method.
  4. Inpainting Occlusions: The most significant challenge addressed is filling the holes caused by occlusions. The authors adapt an existing exemplar-based inpainting method by incorporating height and sensor distance information. This modification allows for a more informed selection of image patches to fill the missing areas. Specifically, the scoring of candidate patches is redefined to consider the distance of the patch center to the sensor and a weighted sum of Sum of Squared Differences (SSD) in both reflectance and height images. This height-aware approach leverages the coupled reflectance/height information and an assumption of alignment between structures to improve the inpainting quality, especially for large occlusions. Finally, a seam-carving method (graphcut) is applied to ensure smooth transitions between inpainted patches.

Datasets

Not discussed in the paper. The paper focuses on a general methodology and demonstrates results on an example orthophoto generated from a mobile LiDAR scan.

Key Insights and Ideas

  • Joint Diffusion of Reflectance and Height: The paper highlights the benefit of diffusing reflectance and height information simultaneously. This coupled approach allows the diffusion process to leverage the complementary information, leading to more coherent and accurate results than processing each channel independently.
  • Height-Aware Inpainting: The adaptation of the inpainting method to incorporate height and sensor distance is a crucial contribution. This insight recognizes that the spatial relationships and structural properties captured by height data can significantly improve the quality of filling missing reflectance information, especially in urban environments where structures often have predictable geometric relationships.
  • Leveraging LiDAR's Strengths: The work effectively demonstrates how mobile LiDAR's high resolution and ground-level perspective can be harnessed to overcome the limitations of traditional aerial/satellite mapping, particularly in producing detailed urban maps.

Key Results

The paper presents qualitative results demonstrating the effectiveness of their pipeline. Figure 1 shows a progression from the raw LiDAR point cloud to the final inpainted orthophoto. The final image (Figure 1(e)) successfully fills large occlusions and preserves structures, resulting in a dense and artifact-free orthophoto. The authors state that the initial image had approximately 2.10⁶ pixels with 31% missing information, which is effectively addressed by their method.

Limitations and Future Work

  • Limitations: The paper acknowledges that the initial projection of LiDAR points results in sparse and noisy images. While their method addresses these issues, the quality of the final output is inherently dependent on the quality and density of the input LiDAR data. The paper also mentions the challenge of large occlusions and how their method attempts to address this by assuming parallel structures to the acquisition path.
  • Future Work: The authors plan to fuse the anisotropic diffusion and denoising steps into a single process and optimize the pipeline for processing larger datasets. This suggests a focus on improving computational efficiency and potentially further refining the integration of the different processing stages.

Summary updated

Abstract

Cet article présente un processus pour la production jointe d'orthophotographie de réflectance et de hauteur à très grande résolution (1cm) à partir d'un levé laser mobile. La projection verticale du nuage de points LiDAR sur une grille de pixels 2D, après filtrage des points du sol, génère deux images éparses et bruitées (réflectance et de hauteur). Nous proposons d'en dériver une image dense par diffusion anisotrope couplée à un débruitage par régularisation TV-L 1. De plus, nous adaptons une méthode d'inpainting afin de combler les zones d'occlutation en tirant parti de la reconstruction couplée réflectance/hauteur et d'un a priori d'alignement entre les structures à inpainter.
BibTeX
@inproceedings{biasutti2016diffusion,
  author = {Pierre Biasutti and Jean–François Aujol and Mathieu Brédif and Aurélie Bugeau},
  title = {{Diffusion anisotrope et inpainting d'orthophotographies LiDAR mobile}},
  booktitle = {HAL},
  year = {2016},
}
Figure