Abstract
Moving object segmentation (MOS) on LiDAR point clouds is crucial for autonomous systems like self-driving vehicles. Previous supervised approaches rely heavily on costly manual annotations, while LiDAR sequences naturally capture temporal motion cues that can be leveraged for self-supervised learning. In this paper, we propose Temporal Overlapping Prediction (TOP), a self-supervised pre-training method that alleviates the labeling burden for MOS. TOP explores the temporal overlapping points commonly observed by current and adjacent scans and learns spatiotemporal representations by predicting the occupancy states of these points. Moreover, we utilize current occupancy reconstruction as an auxiliary pre-training objective, enhancing the model's structural awareness. We conduct extensive experiments and observe that the conventional Intersection-over-Union (IoU) metric shows a strong bias toward objects with more scanned points, potentially neglecting small or distant objects. To compensate, we introduce an additional metric, $\text{mIoU}_{\text{obj}}$, to evaluate object-level performance. Experiments on nuScenes and SemanticKITTI show that TOP outperforms both supervised training-from-scratch and other self-supervised baselines by up to 28.77% relative improvement, demonstrating strong transferability and generalization.
Introduction
For autonomous vehicles, distinguishing between static surroundings and moving obstacles is a fundamental safety requirement. This task, known as Moving Object Segmentation (MOS), typically relies on deep learning models trained via supervised learning. However, manually labeling millions of 3D points as "moving" or "static" is prohibitively expensive and slow.
To address this, researchers have turned to self-supervised pre-training. While previous methods often focused on future frame forecasting, these tasks are frequently more complex than MOS itself, requiring the model to learn intricate flow patterns and reconstruct hidden geometries. This often leads to overfitting and poor generalization across different sensor types. TOP offers a simpler, physics-based alternative that leverages the inherent temporal consistency of LiDAR data.
The TOP Framework
The core principle of TOP is temporal occupancy inconsistency. If a LiDAR beam at one time hits a point in space and a subsequent beam passes through that same point, any change in the "occupancy state" indicates motion.
The TOP pipeline consists of three primary stages:
- Finding Intersections: The system identifies "temporal overlapping points" where LiDAR beams from the current scan intersect with beams from adjacent (past or future) scans.
- Physics-Based Labeling: Utilizing LiDAR beam properties, such as beam divergence, the system automatically labels these intersection points as "Free," "Occupied," or "Unknown."
- Training Objective: A Sparse 4D UNet is trained to predict these occupancy states. An auxiliary task of reconstructing the current scene's geometry is included to ensure the model maintains a strong understanding of static environmental structures.
By focusing on these specific points of intersection, the model learns to recognize the "flicker" of motion without human intervention. The inclusion of beam divergence physics helps the model avoid discretization errors common in voxel-based methods.
Experimental Setup and Metrics
The researchers evaluated TOP using two industry-standard datasets: nuScenes (32-beam LiDAR) and SemanticKITTI (64-beam LiDAR). The latter was specifically used to test cross-dataset transferability.
A key contribution of this work is the critique of standard evaluation metrics. The authors argue that standard IoU is biased toward large objects and the ego-vehicle. To provide a more accurate assessment, they advocate for:
- Object-level Recall ($Recall_{obj}$): To better detect small, vulnerable road users.
- $IoU_{w/o}$: Excluding the car's own points.
- $\text{mIoU}_{\text{obj}}$: A new metric designed to evaluate performance at the object level.
Results and Discussion
TOP consistently outperformed both "training-from-scratch" baselines and state-of-the-art self-supervised methods like 4DOcc and UnO.
- Few-Shot Performance: With only 10% of labeled data, TOP improved object-level recall by 12.17% relative to the baseline.
- Transferability: In cross-dataset tests (training on nuScenes, testing on SemanticKITTI), TOP significantly outperformed competitors, proving it learns general motion principles rather than sensor-specific patterns.
- Generalization: The pre-training also improved performance on static semantic segmentation, indicating a superior grasp of 3D geometry.
Conclusion
TOP demonstrates that a physics-aligned proxy task can significantly improve MOS performance while reducing reliance on manual labels. While the method currently depends on accurate sensor poses for calculating intersections, future work will explore making the framework more robust to localization noise and extending it to tasks like multi-object tracking.
Summary updated