FRNet: Frustum-Range Networks for Scalable LiDAR Segmentation

Xiang Xu, Lingdong Kong, Hui Shuai, Qingshan Liu

Abstract

LiDAR semantic segmentation is critical for autonomous driving, yet existing methods struggle to balance high accuracy with real-time efficiency. Point-based methods offer precision at high computational costs, while range-view methods provide speed but suffer from information loss during projection. We present FRNet (Frustum-Range Network), which bridges this gap using a novel Frustum-Range representation. By grouping points into 3D frustums and utilizing a hierarchical fusion module, FRNet maintains point-level features while leveraging efficient 2D convolutions. FRNet achieves state-of-the-art performance on major benchmarks, including 73.3% mIoU on SemanticKITTI, while operating at up to 33.8 FPS, significantly outperforming existing real-time solutions.

Introduction

LiDAR semantic segmentation is a cornerstone of autonomous driving perception. However, researchers face a persistent trade-off between efficiency and accuracy. Existing methods generally fall into two categories: point/voxel-based methods and range-view methods. Point-based methods operate directly in 3D space, offering high accuracy but failing to meet real-time requirements due to expensive sparse convolutions. Conversely, range-view methods project 3D points onto a 2D range image, allowing for fast 2D CNN processing. However, this projection is "lossy," discarding points due to occlusion and requiring heavy post-processing to recover spatial context. FRNet addresses this scalability challenge by handling massive LiDAR data with high precision without sacrificing millisecond-level speeds.

Methodology

The core innovation of FRNet is the Frustum-Range representation, which treats the 2D range image as a gateway to 3D frustums rather than a flat projection.

Overview of the FRNet architecture, highlighting the Frustum Feature Encoder and the hierarchical Frustum-Point Fusion Module.

Frustum Feature Encoder (FFE)

Instead of selecting a single point per pixel, the FFE groups all points falling into the same 3D frustum. It utilizes Multi-Layer Perceptrons (MLPs) to extract features for every individual point, ensuring that no geometric data is discarded during the initial processing stage.

Frustum-Point (FP) Fusion Module

The FP Fusion Module acts as a hierarchical bridge between 2D and 3D representations. At each stage of the 2D backbone, the model performs two tasks: back-projecting 2D features from the range image to individual 3D points (Frustum-to-Point) and aggregating point-level features back into the 2D representation (Point-to-Frustum). This dual-pathway allows the network to learn global 2D context and local 3D geometry simultaneously.

RangeInterpolation and FrustumMix

To address the inherent sparsity of LiDAR data, the RangeInterpolation module reconstructs semantic surfaces by aggregating information from surrounding valid pixels. Additionally, FrustumMix, a novel data augmentation technique, swaps entire 3D frustum regions between scenes. This preserves the geometric integrity of objects, unlike standard augmentation methods that may disrupt object structure.

Experimental Results

FRNet was evaluated on major benchmarks including SemanticKITTI, nuScenes, ScribbleKITTI, and SemanticPOSS.

Comparison of mIoU vs. FPS on SemanticKITTI. FRNet achieves a superior balance of accuracy and real-time performance compared to point-based and range-view competitors.

Performance and Efficiency

FRNet achieved 73.3% mIoU on SemanticKITTI and 82.5% mIoU on nuScenes, rivaling heavy voxel-based models while maintaining a significant speed advantage. The standard FRNet runs at 29.1 FPS, while the "Fast" variant reaches 33.8 FPS on a single 2080Ti GPU—roughly 5x faster than top-tier voxel methods like Cylinder3D.

Robustness and Label Efficiency

On the Robo3D benchmark, FRNet demonstrated superior resilience to environmental noise such as fog and rain compared to both point-based and range-based competitors. Furthermore, the model maintained high accuracy even with limited labeled data (e.g., 1% or 10% quotas) thanks to the FrustumMix augmentation.

Discussion and Conclusion

FRNet demonstrates that the frustum is a natural mathematical link between 2D pixels and 3D space. By centering the architecture on the frustum, FRNet achieves the efficiency of 2D convolutions without the information loss typical of 2D projections. Because the network maintains point-level features throughout, it can predict labels for every point directly, eliminating the need for computationally expensive K-Nearest Neighbor (KNN) post-processing. While FRNet excels in many areas, it can struggle with very small objects where the semantic signal may be overwhelmed by the majority class within a frustum. Future work will explore extending this Frustum-Range logic to 3D object detection and semantic occupancy prediction.

Summary updated

BibTeX
@article{xu2025frnet,
  author = {Xiang Xu and Lingdong Kong and Hui Shuai and Qingshan Liu},
  title = {{FRNet: Frustum-Range Networks for Scalable LiDAR Segmentation}},
  journal = {IEEE Trans. Image Process.},
  year = {2025},
  url = {https://openreview.net/forum?id=1gBAzyWHTC},
}
Figure