OccFiner: Offboard Occupancy Refinement with Hybrid Propagation

Hao Shi, Song Wang, Jiaming Zhang, Xiaoting Yin, Zhongdao Wang, Zhijian Zhao, Guangming Wang, Jianke Zhu, Kailun Yang, Kaiwei Wang

Introduction

In the field of autonomous driving, 3D Semantic Scene Completion (SSC) is a critical task that involves predicting both the geometry and semantic labels (e.g., vehicle, road, vegetation) for every voxel in a 3D volume. While vision-based systems offer a cost-effective alternative to LiDAR, they frequently suffer from geometric noise due to depth estimation errors and temporal inconsistencies. Onboard models, constrained by real-time processing requirements, often produce "flickering" predictions where objects appear or disappear across frames, and occluded regions remain poorly defined.

Comparison of onboard vs. offboard computation showing how hybrid propagation resolves temporal inconsistencies and fills in occlusions.

OccFiner addresses these challenges by shifting the refinement process offboard. By utilizing the entire driving sequence in a post-processing stage, OccFiner leverages "hindsight" to transform noisy onboard predictions into high-fidelity 3D maps suitable for auto-labeling and high-definition mapping.

Methodology: Hybrid Propagation

OccFiner employs a two-stage Hybrid Propagation strategy designed to decouple and rectify different error modalities: model-induced prediction bias and sensor-induced measurement bias.

Stage 1: Multi-to-Multi Local Propagation

To address model bias, OccFiner introduces the DualFlow4D Transformer. Unlike traditional refinement methods that process frames sequentially, this stage operates on a window of neighboring frames alongside randomly sampled distant reference frames. This "multi-to-multi" approach allows the model to implicitly align features and fill in occlusions by borrowing information from viewpoints where the vehicle had a clearer line of sight. The transformer utilizes soft-patch-based focal attention to maintain spatial consistency while integrating temporal data.

Stage 2: Region-centric Global Propagation

To correct sensor-specific measurement bias, the system employs explicit multi-view geometry. Refined voxels are converted into semantic point clouds and aggregated into a global map using Sensor-Aware Voting. This mechanism recognizes that camera data is most reliable at the center of the Field of View (FOV) and close to the vehicle. By weighting observations based on these physical constraints—trusting proximal data more than distorted peripheral data—the system produces a geometrically consistent global representation.

The two-stage OccFiner pipeline: Multi-to-Multi Local Propagation (Stage 1) followed by Region-centric Global Propagation (Stage 2).

Experimental Results

The framework was evaluated on the SemanticKITTI and SSCBench-KITTI360 benchmarks. OccFiner achieved significant performance gains across various base models:

  • State-of-the-Art Performance: OccFiner currently holds the #1 position on the SemanticKITTI test leaderboard among all vision-based methods.
  • Relative Improvements: When applied to the VoxFormer model, OccFiner delivered a 26.99% relative improvement in Mean Intersection over Union (mIoU).
  • Cross-Modality Efficacy: Although optimized for vision, the refinement logic is modality-agnostic; it improved the LiDAR-based SCPNet by 9.5%.
  • LiDAR Competitiveness: Refined vision-only predictions reached accuracy levels competitive with established LiDAR-based onboard systems like SSCNet.

Discussion and Limitations

OccFiner demonstrates that offboard refinement can effectively "close the data loop" by generating high-quality labels automatically, which can then be used to retrain and improve onboard models. However, some limitations remain. The Stage 1 transformer currently requires individual retraining for specific onboard models, suggesting a need for future research into cross-model generalization. Additionally, while Stage 1 is efficient (0.027s/frame), the Stage 2 global aggregation is computationally intensive, requiring approximately 6 seconds per frame on a CPU, which reinforces its role as a strictly offboard tool.

Conclusion

OccFiner proves that vision-only systems, when augmented with offboard hybrid propagation, can generate 3D occupancy maps that rival LiDAR-based systems. This framework provides a scalable path for high-fidelity map generation and automated data annotation in the autonomous driving industry.

Summary updated

Abstract

Vision-based occupancy prediction, also known as 3D Semantic Scene Completion (SSC), presents a significant challenge in computer vision. Previous methods, confined to onboard processing, struggle with simultaneous geometric and semantic estimation, continuity across varying viewpoints, and single-view occlusion. Our paper introduces OccFiner, a novel offboard framework designed to enhance the accuracy of vision-based occupancy predictions. OccFiner operates in two hybrid phases: 1) a multi-to-multi local propagation network that implicitly aligns and processes multiple local frames for correcting onboard model errors and consistently enhancing occupancy accuracy across all distances. 2) the region-centric global propagation, focuses on refining labels using explicit multi-view geometry and integrating sensor bias, particularly for increasing the accuracy of distant occupied voxels. Extensive experiments demonstrate that OccFiner improves both geometric and semantic accuracy across various types of coarse occupancy, setting a new state-of-the-art performance on the SemanticKITTI dataset. Notably, OccFiner significantly boosts the performance of vision-based SSC models, achieving accuracy levels competitive with established LiDAR-based onboard SSC methods. Furthermore, OccFiner is the first to achieve automatic annotation of SSC in a purely vision-based approach. Quantitative experiments prove that OccFiner successfully facilitates occupancy data loop-closure in autonomous driving. Additionally, we quantitatively and qualitatively validate the superiority of the offboard approach on city-level SSC static maps. The source code will be made publicly available at https://github.com/MasterHow/OccFiner
BibTeX
@article{shi2025occfiner,
  author = {Hao Shi and Song Wang and Jiaming Zhang and Xiaoting Yin and Zhongdao Wang and Zhijian Zhao and Guangming Wang and Jianke Zhu and Kailun Yang and Kaiwei Wang},
  title = {{OccFiner: Offboard Occupancy Refinement with Hybrid Propagation}},
  journal = {IEEE Transactions on Intelligent Transportation Systems},
  year = {2025},
  url = {https://openreview.net/forum?id=Kd0BdOmjm1},
}
Figure