Abstract
Moving object segmentation (MOS) is a task to distinguish moving objects, e.g., moving vehicles and pedestrians, from the surrounding static environment. The segmentation accuracy of MOS can have an influence on odometry, map construction, and planning tasks. In this paper, we propose a semantics-guided convolutional neural network for moving object segmentation. The network takes sequential LiDAR range images as inputs. Instead of segmenting the moving objects directly, the network conducts single-scan-based semantic segmentation and multiple-scan-based moving object segmentation in turn. The semantic segmentation module provides semantic priors for the MOS module, where we propose an adjacent scan association (ASA) module to convert the semantic features of adjacent scans into the same coordinate system to fully exploit the cross-scan semantic features. Finally, by analyzing the difference between the transformed features, reliable MOS result can be obtained quickly. Experimental results on the SemanticKITTI MOS dataset proves the effectiveness of our work.
Introduction
For autonomous vehicles, distinguishing between moving objects (like driving cars or walking pedestrians) and the static environment (buildings, trees, parked cars) is critical for safe path planning and accurate mapping. While camera-based methods are common, LiDAR-based MOS is challenging due to the sparse nature of point clouds and the extreme data imbalance—static objects occupy the vast majority of any given scene.
Existing LiDAR MOS methods often rely on "residual images" (the raw difference between consecutive scans). However, these can be noisy and lack context. Furthermore, there is significantly more labeled data available for general semantic segmentation (identifying what an object is) than for moving object segmentation (identifying if it is currently in motion). This paper introduces a semantics-guided approach to MOS for 3D LiDAR by leveraging semantic features as powerful priors.
Methodology
The proposed pipeline consists of a two-stage, semantics-guided convolutional neural network (CNN). To process 3D data efficiently, LiDAR point clouds are first projected into 2D range images using spherical projection. These images typically include components like range, x, y, z, and intensity.
The architecture is divided into three main components:
- Single-Scan Semantic Segmentation: An off-the-shelf model (e.g., RangeNet++ or SalsaNext) extracts high-level semantic features from individual scans. This provides a massive hint for motion; for instance, a "building" pixel is likely static, whereas a "car" pixel is a candidate for motion.
- Adjacent Scan Association (ASA) Module: Because the sensor platform is moving, consecutive scans are in different coordinate systems. The ASA module uses odometry data to transform the semantic features of a previous scan into the current scan's coordinate frame, allowing for direct spatial comparison.
- Moving Object Segmentation (MOS) Module: This module takes the current semantic features, the transformed previous features, and the current range image as inputs. By analyzing the differences between these aligned features, it predicts which pixels correspond to moving objects.
Experimental Evaluation
The model was evaluated on the SemanticKITTI MOS dataset, which contains 22 sequences of LiDAR scans from outdoor driving scenes. The proposed method achieved an Intersection-over-Union (IoU) of 60.6% on the test set, performing competitively with state-of-the-art methods like KPConv (60.9%) and LMNet (62.5%).
A key advantage is efficiency: while some competing models require up to 8 consecutive scans, this method achieves high accuracy using only two adjacent scans. Ablation studies confirmed that removing semantic features and relying only on raw range data caused the IoU to drop significantly (from ~60% to ~38%), proving that semantic guidance is the primary driver of success.
Conclusion
By splitting the task into "what is it?" (semantic segmentation) and "is it moving?" (MOS), the proposed framework simplifies the training process and improves robustness. Future work will focus on reducing dependency on precise pose estimation and exploring more sophisticated temporal fusion techniques to handle longer sequences without accumulating errors.
Summary updated