Moving Object Segmentation in 3D LiDAR Data: A Learning-based Approach Exploiting Sequential Data

Xieyuanli Chen, Shijie Li, Benedikt Mersch, Louis Wiesmann, Jürgen Gall, Jens Behley, Cyrill StachnissIEEE RA-L 2021 · DOI · OpenReview

Abstract

The ability to detect and segment moving objects in a scene is essential for building consistent maps, making future state predictions, avoiding collisions, and planning. In this paper, we address the problem of moving object segmentation from 3D LiDAR scans. We propose a novel approach that pushes the current state of the art in LiDAR-only moving object segmentation forward to provide relevant information for autonomous robots and other vehicles. Instead of segmenting the point cloud semantically, i.e., predicting the semantic classes such as vehicles, pedestrians, roads, etc., our approach accurately segments the scene into moving and static objects, i.e., also distinguishing between moving cars vs. parked cars. Our proposed approach exploits sequential range images from a rotating 3D LiDAR sensor as an intermediate representation combined with a convolutional neural network and runs faster than the frame rate of the sensor. We compare our approach to several other state-of-the-art methods showing superior segmentation quality in urban environments. Additionally, we created a new benchmark for LiDAR-based moving object segmentation based on SemanticKITTI.

Introduction

In the context of autonomous driving, distinguishing between static and dynamic elements is critical for safe navigation and reliable mapping. While standard LiDAR-based semantic segmentation identifies categories like "car" or "building," it often fails to differentiate between a parked car (a static landmark) and a driving car (a dynamic obstacle).

Overview of the Moving Object Segmentation (MOS) task, showing raw data, range projections, and the final binary segmentation of moving objects.

This paper addresses the challenge of online Moving Object Segmentation (MOS) without requiring prior maps. By processing current and recent LiDAR scans in real-time, the proposed method identifies actually moving objects, enhancing the robustness of SLAM and planning systems.

Methodology

The proposed approach utilizes range images—2D spherical projections of 3D point clouds—as an efficient intermediate representation. The core innovation lies in the generation and use of residual images to capture temporal motion cues.

The proposed pipeline: sequential scans are compensated for ego-motion, projected into range images, and used to calculate residuals which are then processed by a CNN.

The pipeline consists of three main steps:

  1. Ego-motion Compensation: Previous scans are transformed into the current local coordinate system using odometry estimates from a SLAM system.
  2. Re-projection: These transformed scans are projected into the current range image view.
  3. Residual Calculation: The normalized absolute difference between the current and transformed past range values is computed.

These residual images highlight areas where range values have changed significantly, indicating motion. These residuals are concatenated with the current range image and fed into a lightweight CNN (e.g., SalsaNext or RangeNet++) to produce a binary output mask of moving objects.

Datasets and Evaluation

The method was trained and evaluated using SemanticKITTI. To support this research, the authors introduced a New MOS Benchmark, manually reorganizing SemanticKITTI labels into "moving" and "static" classes. This benchmark allows for transparent comparison of MOS-specific performance across different research approaches.

Key Insights

  • Motion vs. Movability: The system focuses on actual motion rather than the potential to move, which is vital for accurate SLAM and mapping.
  • Temporal Cues via Residuals: Providing pre-calculated residuals simplifies the learning task for the CNN, allowing it to focus on motion patterns rather than raw sequence dynamics.
  • Architecture Agnostic: The "residual image" input format is compatible with various state-of-the-art CNN architectures, making it a versatile "plug-and-play" module.

Results

The method achieved a 62.5% Intersection-over-Union (IoU) on the moving class using the SalsaNext architecture. Crucially, the pipeline operates at approximately 20Hz (51ms per scan), which is twice the typical frame rate of LiDAR sensors, ensuring real-time performance. Integration into the SuMa SLAM pipeline demonstrated that masking dynamic objects significantly improves odometry accuracy by preventing corrupted scan matching.

Conclusion

While the approach is robust and fast, it faces challenges with very slow-moving objects and potential ambiguities in residual patterns. Future work will explore tighter integration between MOS and odometry to mutually improve both tasks.

Summary updated

Abstract

The ability to detect and segment moving objects in a scene is essential for building consistent maps, making future state predictions, avoiding collisions, and planning. In this paper, we address the problem of moving object segmentation from 3D LiDAR scans. We propose a novel approach that pushes the current state of the art in LiDAR-only moving object segmentation forward to provide relevant information for autonomous robots and other vehicles. Instead of segmenting the point cloud semantically, i.e., predicting the semantic classes such as vehicles, pedestrians, roads, etc., our approach accurately segments the scene into moving and static objects, i.e., also distinguishing between moving cars vs. parked cars. Our proposed approach exploits sequential range images from a rotating 3D LiDAR sensor as an intermediate representation combined with a convolutional neural network and runs faster than the frame rate of the sensor. We compare our approach to several other state-of-the-art methods showing superior segmentation quality in urban environments. Additionally, we created a new benchmark for LiDAR-based moving object segmentation based on SemanticKITTI. We published it to allow other researchers to compare their approaches transparently and we furthermore published our code.
BibTeX
@article{chen2021moving,
  author = {Xieyuanli Chen and Shijie Li and Benedikt Mersch and Louis Wiesmann and Jürgen Gall and Jens Behley and Cyrill Stachniss},
  title = {{Moving Object Segmentation in 3D LiDAR Data: A Learning-based Approach Exploiting Sequential Data}},
  journal = {IEEE Robotics Autom. Lett.},
  year = {2021},
  doi = {10.1109/lra.2021.3093567},
}
Figure