Introduction
3D occupancy prediction is a critical task for autonomous driving, requiring models to estimate the volumetric occupancy of a scene from 2D camera inputs. However, existing methods often struggle with the "geometric-semantic coupling" bottleneck. When depth estimation is inaccurate, semantic features are projected to incorrect 3D locations, leading to a "garbage in, garbage out" cycle. To compensate, researchers typically resort to massive 3D networks that are too slow for real-time deployment.
The GSD-Occ Framework
The proposed GSD-Occ framework addresses these challenges by disentangling geometric learning from semantic feature extraction. This is achieved through two primary innovations: a Dual-Branch Network (GSDBN) and a Geometric-Semantic Disentangled Learning (GSDL) strategy.
Dual-Branch Network (GSDBN)
The architecture splits the workload into two specialized paths:
- Voxel Branch: This branch focuses on the 3D geometric "skeleton." It utilizes RLK-3DConv (Re-parameterized Large-Kernel 3D Convolution), which uses multiple small dilated kernels during training that are collapsed into a single large kernel for efficient inference. This allows for a wide receptive field to correct geometric errors without the computational overhead of standard large kernels.
- BEV Branch: This branch handles the "flesh" of the prediction—semantic learning and temporal fusion. By operating in 2D Bird’s-Eye View (BEV) space, it significantly reduces memory and computational requirements.
Geometric-Semantic Disentangled Learning (GSDL)
To prevent early-stage depth errors from hindering semantic training, GSDL employs a ground-truth (GT) depth mixup strategy. Initially, the model uses GT depth to project features, allowing the semantic branch to learn object appearances in 3D accurately. Over time, the model transitions to using its own predicted depth, forcing it to learn how to refine imperfect geometric predictions.
Performance and Results
Evaluated on the Occ3D-nuScenes benchmark, GSD-Occ demonstrates a superior trade-off between accuracy and speed.
Key results include:
- Efficiency: Achieving 39.4% mIoU at 20.0 FPS on an NVIDIA A100, GSD-Occ is roughly 3x faster than previous state-of-the-art models like FB-OCC while delivering higher accuracy.
- Memory Footprint: The model requires only ~4.7 GB of VRAM, making it highly suitable for edge deployment on vehicle hardware compared to heavier models like PanoOcc (~12 GB).
Conclusion
GSD-Occ breaks the dependency loop between geometry and semantics in 3D occupancy prediction. By utilizing re-parameterization for geometric refinement and 2D BEV space for semantic processing, it achieves industry-leading real-time performance without sacrificing accuracy. Future work may involve optimizing these modules for lower-power automotive SoCs.
Summary updated