Filling Missing Values Matters for Range Image-Based Point Cloud Segmentation

Bike Chen, Chen Gong, Juha Röning

LiDAR point cloud segmentation (PCS) is the "eyes" of a self-driving car, allowing it to distinguish a cyclist from a sidewalk in real-time. To do this efficiently, researchers often project 3D point clouds into 2D "range images." While this makes processing lightning-fast, it introduces a frustrating problem: missing values.

A new paper, "Filling Missing Values Matters for Range Image-Based Point Cloud Segmentation," argues that these "holes" in the data are more than just a nuisance—they are a major bottleneck for model performance. By rethinking how we project data and fill these gaps, the authors achieve state-of-the-art results with impressive efficiency.

TL;DR

The authors identify that standard projection methods and motion compensation (deskewing) create artificial "holes" in LiDAR range images, breaking object shapes. They propose Scan Unfolding++ (SU++) to project data more cleanly, a range-dependent KNN interpolation (KNNI) to fill remaining gaps, and a high-performance network called FMVNet that beats existing models in the speed-accuracy trade-off.

What Problem Does This Paper Address?

To process massive outdoor point clouds, models often convert 3D points into a 2D grid (a range image). Think of this like unwrapping a 360-degree panoramic photo where each pixel represents the distance to an object.

However, these images are often riddled with black pixels (missing values). The authors point to three culprits:

  1. Projection Overlap: Standard spherical projections cause scan lines to overlap, especially when lasers aren't perfectly spaced.
  2. The "Deskewing" Paradox: To account for a car's motion, we "deskew" scans. While this is great for 3D accuracy, it creates horizontal gaps when projected into 2D.
  3. Sensor Physics: Sometimes a laser beam just doesn't come back—it hits a black car that absorbs the light or flies off into the infinite sky.

These holes make objects look "broken" (e.g., a bicyclist appearing as a floating torso and a wheel), making it incredibly hard for a neural network to learn coherent geometric shapes.

Methods

The authors tackle the problem with a three-pronged approach:

1. Scan Unfolding++ (SU++)

Instead of standard spherical projection, they use "Scan Unfolding." The "++" version introduces a "skewing" method. Essentially, they temporarily "un-do" the motion compensation during the projection phase to ensure points land in a continuous grid, then use a Look-Up Table (LUT) to map them back. This drastically reduces the number of artificial holes created during preprocessing.

2. Range-Dependent KNN Interpolation (KNNI)

For the holes that remain, the authors use a simple but clever interpolation. For any "invalid" pixel, they look at its horizontal neighbors. Crucially, they fill the hole using the neighbor with the smallest range value (the one closest to the sensor). This ensures that foreground objects (like a pedestrian) stay visible and aren't "eaten" by the background (like a distant wall).

3. FMVNet (Filling Missing Values Network)

They designed a new backbone based on the ConvNeXt architecture. Most image-based networks use a "Stem" module that aggressively shrinks the image (e.g., a 4x4 stride). Since LiDAR range images are already short (often only 64 pixels high), this kills the resolution. FMVNet uses a 1x1 convolution in the stem to preserve every bit of vertical detail.

Datasets

The researchers evaluated their methods on three heavy-hitters in the autonomous driving world:

  • SemanticKITTI: The gold standard for large-scale outdoor LiDAR segmentation.
  • SemanticPOSS: A dataset specifically rich in dynamic instances like people and riders.
  • nuScenes: A massive, diverse dataset for urban navigation.

Key Insights and Ideas

  • Preprocessing is as important as the model: The authors showed that simply changing how you project the data (SU++) can boost the performance of existing models (like RangeNet53++ or FIDNet) without changing a single line of the model's code.
  • The "Minimum Range" Rule: In interpolation, picking the closest neighbor is a powerful heuristic. It mimics how human vision works—closer objects occlude further ones.
  • Resolution is Precious: In 2D image tasks, we often downsample to save compute. In LiDAR, where the vertical resolution is already tiny (64 or 128 "pixels"), downsampling is a "performance killer." Preserving resolution in the early layers of the network is vital.

Key Results

  • Performance Boost: On SemanticKITTI, applying SU++ and KNNI to the baseline RangeNet53++ boosted its mean Intersection over Union (mIoU) from 61.5% to 64.4%.
  • State-of-the-Art Trade-off: Their FMVNet achieved 69.0% mIoU on the SemanticKITTI validation set.
  • Real-time Ready: They introduced Fast FMVNet, which maintains a high accuracy (67.4% mIoU) while running at 48 FPS on an NVIDIA A100—well above the 10-20 FPS requirement for real-time robotics.

Limitations and Future Work

While highly effective, the authors note a few areas for improvement:

  • Manual Intervention: Currently, SU++ requires users to know the "laser ID" or "ring number" for each point. While modern sensors provide this, older datasets might require manual labeling or estimation.
  • Hyperparameter Tuning: The authors used a fixed learning rate and standard loss functions; they suggest that a more exhaustive grid search for these parameters could squeeze even more performance out of the model.
  • Beyond Segmentation: The authors believe these "hole-filling" techniques could be applied to other tasks like object detection or LiDAR-based SLAM (Simultaneous Localization and Mapping).

Summary updated

Abstract

Point cloud segmentation (PCS) plays an essential role in robot perception and navigation tasks. To efficiently understand large-scale outdoor point clouds, their range image representation is commonly adopted. This image-like representation is compact and structured, making range image-based PCS models practical. However, undesirable missing values in the range images damage the shapes and patterns of objects. This problem creates difficulty for the models in learning coherent and complete geometric information from the objects. Consequently, the PCS models only achieve inferior performance. Delving deeply into this issue, we find that the use of unreasonable projection approaches and deskewing scans mainly leads to unwanted missing values in the range images. Besides, almost all previous works fail to consider filling in the unexpected missing values in the PCS task. To alleviate this problem, we first propose a new projection method, namely scan unfolding++ (SU++), to avoid massive missing values in the generated range images. Then, we introduce a simple yet effective approach, namely range-dependent $K$-nearest neighbor interpolation ($K$NNI), to further fill in missing values. Finally, we introduce the Filling Missing Values Network (FMVNet) and Fast FMVNet. Extensive experimental results on SemanticKITTI, SemanticPOSS, and nuScenes datasets demonstrate that by employing the proposed SU++ and $K$NNI, existing range image-based PCS models consistently achieve better performance than the baseline models. Besides, both FMVNet and Fast FMVNet achieve state-of-the-art performance in terms of the speed-accuracy trade-off. The proposed methods can be applied to other range image-based tasks and practical applications.
BibTeX
@article{chen2025filling,
  author = {Bike Chen and Chen Gong and Juha Röning},
  title = {{Filling Missing Values Matters for Range Image-Based Point Cloud Segmentation}},
  journal = {IEEE Trans. Intell. Veh.},
  year = {2025},
  url = {https://openreview.net/forum?id=ndiFli1tS1},
}

Loading related content...

Figure