Filling Missing Values Matters for Range Image-Based Point Cloud Segmentation

Bike Chen, Chen Gong, Juha Röning

Abstract

Point cloud segmentation (PCS) plays an essential role in robot perception and navigation tasks. To efficiently understand large-scale outdoor point clouds, their range image representation is commonly adopted. This image-like representation is compact and structured, making range image-based PCS models practical. However, undesirable missing values in the range images damage the shapes and patterns of objects. This problem creates difficulty for the models in learning coherent and complete geometric information from the objects. Consequently, the PCS models only achieve inferior performance. Delving deeply into this issue, we find that the use of unreasonable projection approaches and deskewing scans mainly leads to unwanted missing values in the range images. Besides, almost all previous works fail to consider filling in the unexpected missing values in the PCS task. To alleviate this problem, we first propose a new projection method, namely scan unfolding++ (SU++), to avoid massive missing values in the generated range images. Then, we introduce a simple yet effective approach, namely range-dependent $K$-nearest neighbor interpolation ($K$NNI), to further fill in missing values. Finally, we introduce the Filling Missing Values Network (FMVNet) and Fast FMVNet. Extensive experimental results on SemanticKITTI, SemanticPOSS, and nuScenes datasets demonstrate that by employing the proposed SU++ and $K$NNI, existing range image-based PCS models consistently achieve better performance than the baseline models. Besides, both FMVNet and Fast FMVNet achieve state-of-the-art performance in terms of the speed-accuracy trade-off.

Introduction

LiDAR point cloud segmentation (PCS) is a critical component for autonomous systems, enabling real-time differentiation between various environmental entities such as pedestrians, vehicles, and infrastructure. To achieve the necessary processing speeds for large-scale outdoor environments, 3D point clouds are frequently projected into 2D range images. This representation is computationally efficient but introduces a significant challenge: missing values, or "holes," in the data.

These missing values are not merely noise; they represent a fundamental bottleneck. When objects are projected with gaps, their geometric coherence is lost—for instance, a cyclist might appear as disconnected fragments. This research identifies the primary causes of these gaps—unreasonable projection methods and the artifacts of motion compensation (deskewing)—and proposes a comprehensive framework to mitigate them.

Methodology

Scan Unfolding++ (SU++)

The standard spherical projection often results in overlapping scan lines and gaps, particularly when lidar sensors have non-uniform laser spacing. Furthermore, "deskewing" (compensating for a vehicle's motion during a scan) improves 3D accuracy but exacerbates horizontal gaps in 2D projections.

The proposed Scan Unfolding++ (SU++) method addresses this by temporarily reversing motion compensation during the projection phase. This ensures that points are mapped to a continuous grid. A Look-Up Table (LUT) is then employed to map these points back to their correct spatial coordinates. This approach significantly reduces the density of artificial holes in the resulting range images.

Comparison of projection methods: (a) Spherical Projection shows significant horizontal gaps compared to (b-d) Scan Unfolding++ variants.

Range-Dependent KNN Interpolation (KNNI)

To address remaining gaps caused by sensor physics (e.g., non-reflective surfaces or beams lost to the sky), the authors introduce Range-Dependent K-Nearest Neighbor Interpolation (KNNI). For any invalid pixel, the algorithm examines its horizontal neighbors and fills the gap using the value of the neighbor with the smallest range (i.e., the one closest to the sensor). This heuristic preserves foreground objects, preventing them from being obscured by background data during the interpolation process.

FMVNet Architecture

The Filling Missing Values Network (FMVNet) is designed to maximize the utility of the restored range images. Unlike standard image backbones that use aggressive downsampling (e.g., 4x4 strides) which can destroy the limited vertical resolution of LiDAR data (often only 64 or 128 pixels), FMVNet utilizes a 1x1 convolution in its initial stem. This architectural choice preserves fine-grained vertical details essential for accurate segmentation.

Experiments and Results

The framework was evaluated on three major benchmarks: SemanticKITTI, SemanticPOSS, and nuScenes.

Performance Gains

The integration of SU++ and KNNI provided a consistent performance boost across existing models. For example, the baseline RangeNet53++ saw its mean Intersection over Union (mIoU) on SemanticKITTI increase from 61.5% to 64.4% simply by improving data preprocessing.

Efficiency and Accuracy

The proposed FMVNet achieved a state-of-the-art mIoU of 69.0% on the SemanticKITTI validation set. For real-time applications, Fast FMVNet maintains high accuracy (67.4% mIoU) while operating at 48 FPS on an NVIDIA A100, comfortably exceeding the real-time requirements of autonomous robotics.

Conclusion

This work demonstrates that the quality of the range image representation is as vital as the neural network architecture itself. By addressing the "missing values" problem through improved projection (SU++) and intelligent interpolation (KNNI), significant gains in segmentation accuracy can be achieved. These techniques are model-agnostic and can be integrated into various range image-based tasks, offering a robust path forward for more reliable robot perception.

Summary updated

Abstract

Point cloud segmentation (PCS) plays an essential role in robot perception and navigation tasks. To efficiently understand large-scale outdoor point clouds, their range image representation is commonly adopted. This image-like representation is compact and structured, making range image-based PCS models practical. However, undesirable missing values in the range images damage the shapes and patterns of objects. This problem creates difficulty for the models in learning coherent and complete geometric information from the objects. Consequently, the PCS models only achieve inferior performance. Delving deeply into this issue, we find that the use of unreasonable projection approaches and deskewing scans mainly leads to unwanted missing values in the range images. Besides, almost all previous works fail to consider filling in the unexpected missing values in the PCS task. To alleviate this problem, we first propose a new projection method, namely scan unfolding++ (SU++), to avoid massive missing values in the generated range images. Then, we introduce a simple yet effective approach, namely range-dependent $K$-nearest neighbor interpolation ($K$NNI), to further fill in missing values. Finally, we introduce the Filling Missing Values Network (FMVNet) and Fast FMVNet. Extensive experimental results on SemanticKITTI, SemanticPOSS, and nuScenes datasets demonstrate that by employing the proposed SU++ and $K$NNI, existing range image-based PCS models consistently achieve better performance than the baseline models. Besides, both FMVNet and Fast FMVNet achieve state-of-the-art performance in terms of the speed-accuracy trade-off. The proposed methods can be applied to other range image-based tasks and practical applications.
BibTeX
@article{chen2025filling,
  author = {Bike Chen and Chen Gong and Juha Röning},
  title = {{Filling Missing Values Matters for Range Image-Based Point Cloud Segmentation}},
  journal = {IEEE Trans. Intell. Veh.},
  year = {2025},
  url = {https://openreview.net/forum?id=ndiFli1tS1},
}
Figure