ErpGS: Equirectangular Image Rendering Enhanced with 3D Gaussian Regularization

Shintaro Ito, Natsuki Takama, Koichi Ito, Hwann-Tzong Chen, T. Aoki2025 · DOI

Introduction

Capturing large-scale environments for Virtual Reality (VR), Augmented Reality (AR), or robotics typically requires extensive photographic coverage. 360-degree cameras offer a significant efficiency advantage by capturing the entire surrounding environment in a single shot. However, these cameras output images using Equirectangular Projection (ERP), which introduces severe distortion, particularly near the poles.

Standard 3D Gaussian Splatting (3DGS) algorithms struggle with ERP images, often generating oversized, inaccurate Gaussians that degrade rendering quality. Additionally, 360-degree captures frequently include the camera rig or photographer, which move relative to the scene and disrupt geometric consistency. ErpGS (Omnidirectional Gaussian Splatting) addresses these challenges by adapting 3DGS specifically for the ERP camera model through geometric and scale regularizations, distortion-aware weighting, and dynamic masking.

Methodology: Adapting 3DGS for Omnidirectional Vision

ErpGS introduces several key techniques to mitigate the artifacts inherent in equirectangular 3D reconstruction.

The ErpGS pipeline incorporates omnidirectional Gaussian splatting with geometric and scale regularizations to handle ERP distortion.

Geometric and Scale Regularization

To ensure coherent 3D geometry and prevent "floating" artifacts, ErpGS enforces a consistency between normal vectors rendered from 3D Gaussians and those calculated from the rendered depth map. Unlike standard 2D grids, ErpGS calculates depth-normals on a spherical tangent plane to account for ERP distortion. Furthermore, a scale regularization term penalizes excessively large Gaussians and encourages them to remain flat, preventing the model from using "blobs" to compensate for image stretching.

Distortion-Aware Optimization

A critical insight of ErpGS is that treating an ERP image like a standard perspective photo is insufficient. The stretching at high latitudes means that pixels near the poles represent smaller angular areas than those at the equator. ErpGS introduces distortion-aware weights that prioritize the less-distorted equatorial regions during training, which ablation studies have shown to be the most vital component for maintaining high rendering accuracy.

Viewpoint-Dependent Masking

To handle dynamic obstacles like tripod legs or the photographer, ErpGS employs viewpoint-dependent masks. By ignoring these transient elements during optimization, the model avoids creating conflicting geometry that would otherwise manifest as messy "floaters" in the reconstructed scene.

Experimental Evaluation

The authors evaluated ErpGS across three diverse datasets:

  • OmniBlender: Synthetic indoor and outdoor scenes.
  • Ricoh 360: Real-world outdoor scenes captured via an egocentric spiral path.
  • OmniScenes: Real-world indoor scenes from varied stationary positions.

Quantitative and Qualitative Results

ErpGS consistently outperformed state-of-the-art methods such as EgoNeRF, ODGS, and OmniGS. In the "barbershop" scene from OmniBlender, ErpGS achieved a PSNR of 38.71 dB, significantly higher than OmniGS (37.26 dB) and EgoNeRF (30.57 dB).

Qualitative comparison showing ErpGS's ability to eliminate artifacts and maintain sharp details compared to existing methods across various datasets.

As shown in the visual comparisons, ErpGS successfully eliminates the cloudy artifacts and geometric inconsistencies common in other methods, particularly in regions previously occupied by camera hardware. Ablation tests confirmed the necessity of each component: removing distortion weights caused the most significant performance drop (from 35.16 dB to 33.86 dB on OmniBlender), while removing regularizations also led to measurable fidelity losses.

Conclusion and Future Directions

ErpGS represents a significant step forward in photorealistic 3D reconstruction from panoramic imagery. By explicitly modeling the spherical nature of ERP data, it overcomes the limitations of standard 3DGS.

However, the method remains dependent on the quality of initial Structure-from-Motion (SfM) point clouds. In expansive outdoor scenes, such as the 'lone-monk' dataset, ErpGS may fail to render distant objects if the initial sparse point cloud is insufficient. Future research will likely focus on improving geometry initialization and optimization for distant background elements where sparse data is lacking.

Summary updated

Abstract

The use of multi-view images acquired by a 360-degree camera can reconstruct a 3D space with a wide area. There are 3D reconstruction methods from equirectangular images based on NeRF and 3DGS, as well as Novel View Synthesis (NVS) methods. On the other hand, it is necessary to overcome the large distortion caused by the projection model of a 360-degree camera when equirectangular images are used. In 3DGS-based methods, the large distortion of the 360-degree camera model generates extremely large 3D Gaussians, resulting in poor rendering accuracy. We propose ErpGS, which is Omnidirectional GS based on 3DGS to realize NVS addressing the problems. ErpGS introduce some rendering accuracy improvement techniques: geometric regularization, scale regularization, and distortion-aware weights and a mask to suppress the effects of obstacles in equirectangular images. Through experiments on public datasets, we demonstrate that ErpGS can render novel view images more accurately than conventional methods.
BibTeX
@misc{ito2025erpgs,
  author = {Shintaro Ito and Natsuki Takama and Koichi Ito and Hwann-Tzong Chen and T. Aoki},
  title = {{ErpGS: Equirectangular Image Rendering Enhanced with 3D Gaussian Regularization}},
  year = {2025},
  doi = {10.1109/icip55913.2025.11084349},
}
Figure