Achieving Speed-Accuracy Balance in Vision-based 3D Occupancy Prediction via Geometric-Semantic Disentanglement

Yulin He, Wei Chen, Siqi Wang, Tianci Xun, Yusong TanAAAI 2025 · DOI

Introduction

3D occupancy prediction is a critical task for autonomous driving, requiring models to estimate the volumetric occupancy of a scene from 2D camera inputs. However, existing methods often struggle with the "geometric-semantic coupling" bottleneck. When depth estimation is inaccurate, semantic features are projected to incorrect 3D locations, leading to a "garbage in, garbage out" cycle. To compensate, researchers typically resort to massive 3D networks that are too slow for real-time deployment.

The GSD-Occ Framework

The proposed GSD-Occ framework addresses these challenges by disentangling geometric learning from semantic feature extraction. This is achieved through two primary innovations: a Dual-Branch Network (GSDBN) and a Geometric-Semantic Disentangled Learning (GSDL) strategy.

The GSD-Occ architecture featuring the Dual-Branch Network and the GSDL strategy.

Dual-Branch Network (GSDBN)

The architecture splits the workload into two specialized paths:

  1. Voxel Branch: This branch focuses on the 3D geometric "skeleton." It utilizes RLK-3DConv (Re-parameterized Large-Kernel 3D Convolution), which uses multiple small dilated kernels during training that are collapsed into a single large kernel for efficient inference. This allows for a wide receptive field to correct geometric errors without the computational overhead of standard large kernels.
  2. BEV Branch: This branch handles the "flesh" of the prediction—semantic learning and temporal fusion. By operating in 2D Bird’s-Eye View (BEV) space, it significantly reduces memory and computational requirements.

Geometric-Semantic Disentangled Learning (GSDL)

To prevent early-stage depth errors from hindering semantic training, GSDL employs a ground-truth (GT) depth mixup strategy. Initially, the model uses GT depth to project features, allowing the semantic branch to learn object appearances in 3D accurately. Over time, the model transitions to using its own predicted depth, forcing it to learn how to refine imperfect geometric predictions.

Performance and Results

Evaluated on the Occ3D-nuScenes benchmark, GSD-Occ demonstrates a superior trade-off between accuracy and speed.

Comparison of mIoU vs. FPS on the Occ3D-nuScenes benchmark. GSD-Occ (Ours) achieves a high mIoU while maintaining real-time performance.

Key results include:

  • Efficiency: Achieving 39.4% mIoU at 20.0 FPS on an NVIDIA A100, GSD-Occ is roughly 3x faster than previous state-of-the-art models like FB-OCC while delivering higher accuracy.
  • Memory Footprint: The model requires only ~4.7 GB of VRAM, making it highly suitable for edge deployment on vehicle hardware compared to heavier models like PanoOcc (~12 GB).

Conclusion

GSD-Occ breaks the dependency loop between geometry and semantics in 3D occupancy prediction. By utilizing re-parameterization for geometric refinement and 2D BEV space for semantic processing, it achieves industry-leading real-time performance without sacrificing accuracy. Future work may involve optimizing these modules for lower-power automotive SoCs.

Summary updated

BibTeX
@inproceedings{he2025achieving,
  author = {Yulin He and Wei Chen and Siqi Wang and Tianci Xun and Yusong Tan},
  title = {{Achieving Speed-Accuracy Balance in Vision-based 3D Occupancy Prediction via Geometric-Semantic Disentanglement}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2025},
  doi = {10.1609/aaai.v39i3.32358},
}
Figure