LodoNet: A Deep Neural Network with 2D Keypoint Matching for 3D LiDAR Odometry Estimation

Ce Zheng, Yecheng Lyu, Ming Li, Ziming Zhang

TL;DR

This paper introduces LodoNet, a novel deep learning approach for 3D LiDAR odometry that leverages 2D image feature extraction techniques (SIFT) on projected LiDAR point clouds. By transforming the 3D problem into a 2D image-based feature matching task, LodoNet achieves state-of-the-art or comparable performance on the KITTI odometry benchmark, offering a computationally efficient and robust solution.

What Problem Does This Paper Address?

Accurate odometry estimation – determining a vehicle's precise position and orientation over time – is crucial for autonomous driving and robotics. While various sensors like cameras and LiDAR are used, LiDAR offers advantages in distance accuracy and robustness to lighting conditions. Existing LiDAR odometry methods often fall into two categories: traditional point cloud registration (like ICP and LOAM), which struggle with local patterns and can be computationally expensive, or deep learning approaches that directly process large 3D point clouds, also posing computational challenges. This paper aims to bridge this gap by adapting successful image-based feature extraction methods to the LiDAR domain, seeking a more efficient and accurate odometry solution.

Methods

LodoNet's core innovation lies in its novel approach to feature extraction and matching. Instead of directly processing 3D point clouds, the authors first project the sparse LiDAR data onto a spherical image representation. This projection, similar to techniques used in RangeNet++, helps to create a more dense, image-like feature map.

Key steps include:

  • Depth Spherical Projection and Completion: LiDAR point clouds are projected onto a spherical surface. To address sparsity, a depth completion method fills in missing depth values by using the nearest valid pixel's depth.
  • Keypoint Detection and Matching (SIFT): The Scale-Invariant Feature Transform (SIFT) algorithm, a well-established image feature descriptor, is applied to these 2D spherical depth images to detect and match keypoints. This is a significant departure from typical 3D point cloud processing.
  • Projection Back to 3D: The matched 2D keypoint pairs (MKPs) are then projected back into the 3D LiDAR space using an inverse projection function.
  • LodoNet Architecture: A convolutional neural network (CNN) pipeline is designed to estimate odometry from these 3D MKPs. This architecture includes:
  • MKPs Selection Module: This module, inspired by PointNet, filters out potentially unreliable MKPs (e.g., from dynamic objects) to improve robustness.
  • Rotation Estimation Module: This module predicts the 3-DoF relative rotation using a regression model, incorporating a data augmentation strategy for rotation to improve performance.
  • Translation Estimation Module: This module predicts the 3-DoF relative translation.

The paper emphasizes the decoupling of the keypoint matching function (g) and the odometry regression function (h), which helps to avoid the coupled optimization issues found in some ICP-based learning methods and improves convergence.

Datasets

The authors utilize the KITTI Odometry dataset for training and evaluation. Specifically, they use sequences 0, 1, 2, 3, 4, 5, 6, 9, and 10 for training, and sequences 7 and 8 for validation. The KITTI dataset is a standard benchmark for autonomous driving tasks, providing synchronized camera images, LiDAR scans, and ground truth trajectories.

Key Insights and Ideas

  • Leveraging 2D Image Features for 3D LiDAR: The most significant insight is the successful transfer of image-based feature extraction techniques (SIFT) to the LiDAR domain by projecting point clouds into spherical images. This allows for the use of mature and robust image processing tools.
  • Decoupling Feature Matching and Odometry Regression: Separating the keypoint extraction (g) from the odometry estimation (h) simplifies the learning process, improves convergence, and avoids the computational complexities of tightly coupled optimization.
  • Depth Completion for Enhanced Feature Extraction: Filling in sparse depth information in spherical images allows for more comprehensive keypoint detection and matching, leading to richer feature sets for odometry estimation.
  • MKP Selection for Robustness: Explicitly filtering matched keypoints to remove those likely associated with dynamic objects is crucial for improving the accuracy and reliability of odometry estimation, as these points can introduce inconsistent motion information.

Key Results

LodoNet achieves competitive results on the KITTI odometry estimation benchmark. The paper reports performance comparable to or even better than state-of-the-art methods on several sequences. For instance, in Table 1, LodoNet's t_rel and r_rel values are often among the lowest, indicating high accuracy in both translation and rotation estimation. The authors highlight that while LOAM remains a strong geometry-based approach, deep learning methods like LodoNet are becoming increasingly accurate and efficient. Ablation studies demonstrate the effectiveness of rotation augmentation and the selection of 100 MKPs for optimal performance.

Limitations and Future Work

The authors acknowledge that while their method is competitive, LOAM still holds the title of the state-of-the-art geometry-based approach. They also note that some MKPs might still be mismatched or unsuitable for odometry estimation, even after the selection module.

Future work directions suggested include:

  • Further integration of MKP extraction and odometry estimation: Exploring end-to-end learning where both feature extraction and odometry prediction are jointly optimized could lead to even more accurate and efficient estimators.
  • Deployment on automated driving systems: The paper suggests that LodoNet's architecture, being a combination of a conventional feature descriptor and a lightweight neural network, is well-suited for deployment on resource-constrained autonomous driving systems.

Summary updated

Abstract

Deep learning based LiDAR odometry (LO) estimation attracts increasing research interests in the field of autonomous driving and robotics. Existing works feed consecutive LiDAR frames into neural networks as point clouds and match pairs in the learned feature space. In contrast, motivated by the success of image based feature extractors, we propose to transfer the LiDAR frames to image space and reformulate the problem as image feature extraction. With the help of scale-invariant feature transform (SIFT) for feature extraction, we are able to generate matched keypoint pairs (MKPs) that can be precisely returned to the 3D space. A convolutional neural network pipeline is designed for LiDAR odometry estimation by extracted MKPs. The proposed scheme, namely LodoNet, is then evaluated in the KITTI odometry estimation benchmark, achieving on par with or even better results than the state-of-the-art.
BibTeX
@inproceedings{zheng2020lodonet,
  author = {Ce Zheng and Yecheng Lyu and Ming Li and Ziming Zhang},
  title = {{LodoNet: A Deep Neural Network with 2D Keypoint Matching for 3D LiDAR Odometry Estimation}},
  booktitle = {ACM Multimedia},
  year = {2020},
  url = {https://openreview.net/forum?id=IZ5twgWqWL},
}
Figure