Abstract
3D Panoptic Occupancy Prediction (3DPOP) is a fundamental task for autonomous driving perception, aiming to reconstruct dense 3D volumetric maps with both semantic and instance-level information. Traditional methods relying on discrete depth bins often produce jagged geometry and blurry object boundaries. We propose HyGE-Occ, a novel framework that combines discrete depth modeling with continuous 3D Gaussian representations and introduces an explicit edge-prediction task. Evaluated on the Occ3D-nuScenes benchmark, HyGE-Occ achieves state-of-the-art results, demonstrating superior geometric smoothness and instance separation.
Introduction: The Challenge of Blurry Boundaries
In the context of autonomous driving, 3D Panoptic Occupancy Prediction requires assigning every 3D voxel a semantic class and a unique instance ID. Most current architectures utilize the Lift-Splat-Shoot (LSS) paradigm to transform 2D image features into a 3D Bird’s-Eye View (BEV) representation. However, LSS-based methods typically rely on discrete depth bins, which introduce quantization artifacts—often referred to as the "staircase effect." This leads to coarse geometry and "label bleeding," where the boundaries between adjacent objects (e.g., two parked cars) become indistinct.
Methodology
HyGE-Occ addresses these limitations through two core innovations: a hybrid view-transformation branch and an auxiliary edge prediction module.
Hybrid View-Transformation Branch
Rather than relying solely on discrete or continuous depth estimation, HyGE-Occ leverages the strengths of both:
- Discrete Branch: Maintains precise local spatial localization using standard LSS-style unprojection into depth bins.
- Continuous Branch: Employs 3D Gaussian Splatting. By predicting Gaussian parameters (mean, variance, and opacity) for each pixel, the model represents depth as a smooth, continuous distribution. This effectively handles depth uncertainty and interpolates the gaps inherent in discrete binning.
- Feature Blending: The features from both branches are merged via an $\alpha$-blending strategy, resulting in a "Hybrid BEV" feature map that is both spatially accurate and geometrically smooth.
Edge Prediction Module
To further mitigate label bleeding, HyGE-Occ treats boundary detection as an explicit structural prior. The system generates "pseudo-edge" labels from ground-truth semantic maps using a Sobel filter. An auxiliary Edge Head is then trained to predict these edges directly from the BEV features. This forces the network to prioritize high-contrast structural cues, which are essential for separating distinct instances in the final 3D output.
Experimental Results
The framework was rigorously tested on the Occ3D-nuScenes dataset, which includes 700 training and 150 validation scenes with 17 semantic classes.
Performance Metrics
HyGE-Occ established new state-of-the-art benchmarks:
- Accuracy: Reached 39.9 RayIoU and 32.0 mIoU, significantly outperforming baselines like Panoptic-FlashOcc.
- Panoptic Quality: Achieved a RayPQ of 17.5, demonstrating a substantial improvement in instance separation compared to SparseOcc (14.1).
- Efficiency: The "tiny" variant of the model operates at 37.7 FPS, proving its suitability for real-time deployment on vehicle hardware.
Qualitative Improvements
Visual analysis confirms that HyGE-Occ produces significantly cleaner boundaries and fewer "floating" artifacts. The integration of edge priors allows the model to maintain sharp transitions between objects and the background, solving the common issue of blurred semantic labels in dense 3D volumes.
Conclusion
HyGE-Occ demonstrates that discrete and continuous depth representations are complementary. By fusing these approaches and incorporating explicit edge priors, the model achieves a new level of precision in 3D Panoptic Occupancy Prediction. Future work will explore extending this hybrid representation to temporal sequences (4D occupancy) to better capture dynamic scene evolution.
Summary updated