TL;DR
This paper introduces a novel unsupervised method for segmenting moving objects in 4D LiDAR data from stationary sensors. By representing voxel occupancy changes as multivariate time series (MOTS), the approach effectively learns motion patterns without requiring labeled data, achieving performance comparable to supervised methods.
What Problem Does This Paper Address?
Accurately identifying moving objects (moving object segmentation or MOS) in 3D LiDAR point clouds is crucial for applications like autonomous driving, smart city infrastructure, and pedestrian safety. However, existing state-of-the-art deep learning methods for LiDAR MOS heavily rely on expensive and scarce annotated ground truth data. This dependency limits their applicability to scenarios where data is collected with the exact same sensor setup and annotation style. The paper aims to bridge this gap by developing an unsupervised approach that can generalize to different stationary LiDAR sensors and settings without needing any manual annotations.
Methods
The core innovation of this paper lies in its novel representation of 4D LiDAR data: Multivariate Occupancy Time Series (MOTS). Instead of directly processing raw point clouds, the authors first voxelize the 4D LiDAR data. For each voxel, they construct an Occupancy Time Series (OTS) that captures its occupied or free state over a short temporal window. This OTS is then expanded into a MOTS by including the OTS of neighboring voxels within a defined spatial radius. This MOTS effectively encodes spatio-temporal occupancy changes around a voxel.
The unsupervised MOS problem is then reframed as a multivariate time series clustering problem. A 1D Convolutional Neural Network (CNN) autoencoder is trained in a self-supervised manner to learn a latent feature representation (embedding) for each MOTS. This encoder is trained to reconstruct the input MOTS, forcing it to learn meaningful features that distinguish between moving and stationary patterns. Finally, a Gaussian Mixture Model (GMM) is used to cluster these learned voxel embeddings, thereby segmenting the voxels into moving or stationary categories without any supervision.
Datasets
- Raw KITTI dataset [16]: Used for quantitative evaluation. The authors manually annotated three stationary scenes from this dataset.
- SemanticKITTI MOS dataset [3, 7]: Used for training the autoencoder model. The authors note that while this dataset has annotations, it has limitations for evaluating stationary scenarios.
- Custom Velodyne VLP-16 LiDAR data: Collected by the authors for qualitative evaluation on a different sensor.
Key Insights and Ideas
- Reframing MOS as Time Series Clustering: The paper successfully demonstrates that by transforming the problem into a time series clustering task using MOTS, unsupervised segmentation becomes feasible and effective.
- MOTS as a Powerful Representation: The MOTS representation effectively captures local spatio-temporal occupancy changes, which are key indicators of motion, even in stationary sensor setups.
- Self-Supervised Learning for Feature Extraction: Leveraging a 1D CNN autoencoder for self-supervised learning on MOTS proves to be an efficient way to extract discriminative features for motion segmentation.
- Generalization to Different Sensors: The approach shows promise in generalizing to different LiDAR sensors (e.g., HDL-64E and VLP-16) and temporal resolutions, highlighting its robustness.
- Potential for Privacy-Preserving Surveillance: The authors suggest that LiDAR-based MOS, being less privacy-invasive than camera-based methods, could be a valuable tool for smart city surveillance.
Key Results
- Comparable Performance to Supervised Methods: The proposed unsupervised approach achieves comparable performance to state-of-the-art supervised MOS methods on the Raw KITTI dataset, with an average mIoU of 0.812, outperforming the supervised 4DMOS method (0.781).
- Superiority in Certain Scenarios: The method outperforms supervised approaches on the "City 1" and "City 2" scenes, suggesting better generalization to stationary settings where supervised models might struggle due to biased training data.
- Hyperparameter Sensitivity: The study shows that the number of clusters (k), radius (r), embedding dimension (e), and window size (w) significantly influence performance, with optimal settings identified through ablation studies. For instance, a radius of r=2, embedding dimension e=32, and window sizes w=15-20 generally yield the best results.
- Qualitative Success: Qualitative results on VLP-16 data demonstrate accurate segmentation of pedestrians and a cyclist, even with different sensor characteristics.
Limitations and Future Work
- Limitations in Non-Stationary Settings: The current approach is primarily designed for stationary LiDAR sensors. Points entering the scene or becoming visible after occlusion are treated similarly to moving objects, posing a challenge for non-stationary scenarios.
- Receptive Field Limitations: The method is limited by its receptive field, which can be computationally expensive to increase quadratically. Future work aims to find more efficient ways to scale the receptive field.
- Extension to Non-Stationary LiDAR: The authors explicitly state that extending the approach to handle non-stationary LiDAR data is a key direction for future research.
- Efficient Scaling of Receptive Field: Developing more efficient methods for increasing the receptive field of the MOTS representation is also a priority.
Summary updated