Semi-Local Convolutions for LiDAR Scan Processing

Larissa T. Triess, David Peter, J. Marius ZöllnerarXiv 2021 · OpenReview

TL;DR

This paper introduces "Semi-Local Convolutions" (SLC) for LiDAR scan processing, aiming to address the varying appearance of LiDAR data along the vertical axis. Surprisingly, their experiments found that SLCs did not improve semantic segmentation performance compared to standard convolutional layers, suggesting that existing CNNs with sufficient capacity can already handle these variations.

What Problem Does This Paper Address?

Autonomous vehicles and mobile robots rely heavily on LiDAR sensors to perceive their 3D surroundings. To process this data efficiently, many state-of-the-art methods project LiDAR scans into an image-like format and apply deep convolutional neural networks (CNNs) for tasks like semantic segmentation. CNNs typically assume spatial stationarity – that features learned in one part of an image are useful elsewhere. However, projected LiDAR scans can exhibit significant variations in appearance along the vertical axis due to sensor characteristics and the nature of 3D environments. This paper investigates whether this vertical axis variation breaks the spatial stationarity assumption for convolutions and if a modified convolution layer can improve performance.

Methods

The core contribution is the Semi-Local Convolution (SLC) layer. In a standard 2D convolution, filter weights are shared across all spatial dimensions (height and width). SLC modifies this by introducing a parameter, a, which controls the amount of weight sharing along the vertical dimension. Instead of a single filter for the entire vertical extent, SLC uses a different filter components, each responsible for a portion of the vertical axis. When a=1, it's a standard convolution. When a=H_x (the total height of the input), there's no weight sharing along the vertical axis at all, meaning each row gets its own filter. The authors propose this as a way to explicitly model the varying statistics along the vertical axis.

Datasets

The experiments were conducted on the SemanticKITTI dataset, specifically its validation split. This dataset is a benchmark for semantic scene understanding from LiDAR sequences. The input to the networks consisted of a two-channel image representing projected depth measurements and reflectivity values.

Key Insights and Ideas

  • Vertical Axis Variation is Not Necessarily a Problem: The most striking finding is that explicitly modeling the vertical axis variation with SLCs did not yield improvements. This suggests that standard CNNs, with adequate capacity, are already capable of learning these variations.
  • Capacity Matters More Than Specialization: The paper implies that the inherent ability of CNNs to learn spatial location information (as suggested by prior work) is sufficient. If a network has enough parameters and layers, it can learn to adapt to different statistical properties across the vertical dimension without a specialized layer.
  • Ablation Study is Crucial: The authors emphasize the importance of their independent ablation study. By testing SLCs without other model modifications, they aim to isolate the effect of this specific architectural change, making their evaluation more transparent than previous works that combined similar ideas with other network changes.

Key Results

  • No Performance Improvement: In all experiments, using SLC layers did not outperform standard convolution layers in terms of semantic segmentation accuracy or mean Intersection over Union (mIoU).
  • Performance Degradation with Increased a: As the number of vertical filter components (a) increased (meaning less weight sharing vertically), performance generally decreased. This trend was observed across different network architectures (DarkNet21 and SqueezeSegV2).
  • Parameter Scaling: Increasing a also increases the number of trainable parameters. Experiments were conducted to control for this by scaling down the number of filters in other layers, but the performance degradation with increasing a persisted.

Limitations and Future Work

  • Data Dependency: The authors acknowledge that their findings might be data-dependent. Future work could involve evaluating SLCs on other datasets (e.g., nuScenes, Waymo Open) or using different input representations (e.g., dense depth instead of range images).
  • Task Dependency: The results might also be task-specific. It's suggested that SLCs could potentially yield different results for tasks like object detection or motion estimation.
  • Soft SLC: A future direction proposed is a "soft" version of SLC, where the vertical strips are not processed entirely independently. Instead, the output could be a linear combination of multiple kernel operations, weighted by the vertical position.

Summary updated

Abstract

A number of applications, such as mobile robots or automated vehicles, use LiDAR sensors to obtain detailed information about their three-dimensional surroundings. Many methods use image-like projections to efficiently process these LiDAR measurements and use deep convolutional neural networks to predict semantic classes for each point in the scan. The spatial stationary assumption enables the usage of convolutions. However, LiDAR scans exhibit large differences in appearance over the vertical axis. Therefore, we propose semi local convolution (SLC), a convolution layer with reduced amount of weight-sharing along the vertical dimension. We are first to investigate the usage of such a layer independent of any other model changes. Our experiments did not show any improvement over traditional convolution layers in terms of segmentation IoU or accuracy.
BibTeX
@misc{triess2021semi,
  author = {Larissa T. Triess and David Peter and J. Marius Zöllner},
  title = {{Semi-Local Convolutions for LiDAR Scan Processing}},
  year = {2021},
  eprint = {2111.15615},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2111.15615},
}
Figure