HyGE-Occ: Hybrid View-Transformation with 3D Gaussian and Edge Priors for 3D Panoptic Occupancy Prediction

Jong Wook Kim, Wonseok Roh, Hadam Baek, Pilhyeon Lee, Jonghyun Choi, Sangpil KimarXiv 2025 · DOI · arXiv

Abstract

3D Panoptic Occupancy Prediction (3DPOP) is a fundamental task for autonomous driving perception, aiming to reconstruct dense 3D volumetric maps with both semantic and instance-level information. Traditional methods relying on discrete depth bins often produce jagged geometry and blurry object boundaries. We propose HyGE-Occ, a novel framework that combines discrete depth modeling with continuous 3D Gaussian representations and introduces an explicit edge-prediction task. Evaluated on the Occ3D-nuScenes benchmark, HyGE-Occ achieves state-of-the-art results, demonstrating superior geometric smoothness and instance separation.

Introduction: The Challenge of Blurry Boundaries

In the context of autonomous driving, 3D Panoptic Occupancy Prediction requires assigning every 3D voxel a semantic class and a unique instance ID. Most current architectures utilize the Lift-Splat-Shoot (LSS) paradigm to transform 2D image features into a 3D Bird’s-Eye View (BEV) representation. However, LSS-based methods typically rely on discrete depth bins, which introduce quantization artifacts—often referred to as the "staircase effect." This leads to coarse geometry and "label bleeding," where the boundaries between adjacent objects (e.g., two parked cars) become indistinct.

Conceptual overview of HyGE-Occ, combining continuous and discrete representations with edge priors for panoptic occupancy prediction.

Methodology

HyGE-Occ addresses these limitations through two core innovations: a hybrid view-transformation branch and an auxiliary edge prediction module.

Hybrid View-Transformation Branch

Rather than relying solely on discrete or continuous depth estimation, HyGE-Occ leverages the strengths of both:

  • Discrete Branch: Maintains precise local spatial localization using standard LSS-style unprojection into depth bins.
  • Continuous Branch: Employs 3D Gaussian Splatting. By predicting Gaussian parameters (mean, variance, and opacity) for each pixel, the model represents depth as a smooth, continuous distribution. This effectively handles depth uncertainty and interpolates the gaps inherent in discrete binning.
  • Feature Blending: The features from both branches are merged via an $\alpha$-blending strategy, resulting in a "Hybrid BEV" feature map that is both spatially accurate and geometrically smooth.

Edge Prediction Module

To further mitigate label bleeding, HyGE-Occ treats boundary detection as an explicit structural prior. The system generates "pseudo-edge" labels from ground-truth semantic maps using a Sobel filter. An auxiliary Edge Head is then trained to predict these edges directly from the BEV features. This forces the network to prioritize high-contrast structural cues, which are essential for separating distinct instances in the final 3D output.

Detailed architecture of HyGE-Occ, showing the Hybrid View-Transformation Branch, Edge Prediction Module, and Panoptic Head.

Experimental Results

The framework was rigorously tested on the Occ3D-nuScenes dataset, which includes 700 training and 150 validation scenes with 17 semantic classes.

Performance Metrics

HyGE-Occ established new state-of-the-art benchmarks:

  • Accuracy: Reached 39.9 RayIoU and 32.0 mIoU, significantly outperforming baselines like Panoptic-FlashOcc.
  • Panoptic Quality: Achieved a RayPQ of 17.5, demonstrating a substantial improvement in instance separation compared to SparseOcc (14.1).
  • Efficiency: The "tiny" variant of the model operates at 37.7 FPS, proving its suitability for real-time deployment on vehicle hardware.

Qualitative Improvements

Visual analysis confirms that HyGE-Occ produces significantly cleaner boundaries and fewer "floating" artifacts. The integration of edge priors allows the model to maintain sharp transitions between objects and the background, solving the common issue of blurred semantic labels in dense 3D volumes.

Conclusion

HyGE-Occ demonstrates that discrete and continuous depth representations are complementary. By fusing these approaches and incorporating explicit edge priors, the model achieves a new level of precision in 3D Panoptic Occupancy Prediction. Future work will explore extending this hybrid representation to temporal sequences (4D occupancy) to better capture dynamic scene evolution.

Summary updated

Abstract

3D Panoptic Occupancy Prediction aims to reconstruct a dense volumetric scene map by predicting the semantic class and instance identity of every occupied region in 3D space. Achieving such fine-grained 3D understanding requires precise geometric reasoning and spatially consistent scene representation across complex environments. However, existing approaches often struggle to maintain precise geometry and capture the precise spatial range of 3D instances critical for robust panoptic separation. To overcome these limitations, we introduce HyGE-Occ, a novel framework that leverages a hybrid view-transformation branch with 3D Gaussian and edge priors to enhance both geometric consistency and boundary awareness in 3D panoptic occupancy prediction. HyGE-Occ employs a hybrid view-transformation branch that fuses a continuous Gaussian-based depth representation with a discretized depth-bin formulation, producing BEV features with improved geometric consistency and structural coherence. In parallel, we extract edge maps from BEV features and use them as auxiliary information to learn edge cues. In our extensive experiments on the Occ3D-nuScenes dataset, HyGE-Occ outperforms existing work, demonstrating superior 3D geometric reasoning.
BibTeX
@misc{kim2025hyge,
  author = {Jong Wook Kim and Wonseok Roh and Hadam Baek and Pilhyeon Lee and Jonghyun Choi and Sangpil Kim},
  title = {{HyGE-Occ: Hybrid View-Transformation with 3D Gaussian and Edge Priors for 3D Panoptic Occupancy Prediction}},
  year = {2025},
  eprint = {2512.19871},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2512.19871},
}
Figure