Introduction
360° depth perception is a cornerstone for autonomous navigation in indoor environments. While LiDAR provides high accuracy, its cost and mechanical complexity often favor camera-based solutions. Fisheye cameras, in particular, offer a wide field of view (FOV) but introduce significant radial distortion that challenges standard Convolutional Neural Networks (CNNs). This paper presents a novel approach to 360° depth estimation using a plane-sweeping stereo pipeline based on an icosahedron representation, termed "CrownConv."
Methodology
The core of the proposed method is the decoupling of the camera hardware from the neural network weights by projecting raw fisheye images onto a standardized icosahedron grid.
Icosahedron-based Representation
Instead of using equirectangular projections which suffer from severe distortion at the poles, the authors utilize an icosahedron—a 20-sided polyhedron that provides a more uniform sampling of the sphere. To process this data efficiently with standard 2D CNNs, the icosahedron is unfolded into a specific layout resembling an "origami crown." This "CrownConv" representation allows the use of optimized 2D convolutions while maintaining the global context of the spherical environment.
Icospherical Sweeping and Depth Regression
The pipeline employs a plane-sweeping stereo approach adapted for spherical geometry. Virtual spheres are swept at varying radii to construct a cost volume directly on the icosahedron grid. This cost volume is regularized using a 3D version of CrownConv, and the final continuous depth map is extracted via a soft argmax function. By aligning the icosahedron with the gravity vector using extrinsic camera parameters, the system remains robust to variations in physical camera mounting.
Experiments and Results
The model was evaluated on synthetic datasets (OmniThings and OmniHouse) and a newly introduced "Rotated OmniHouse" dataset designed to test robustness against camera misalignment.
Robustness to Alignment
A key finding is the system's resilience to changes in the camera rig. While traditional models like OmniMVS saw their Root Mean Square (RMS) error more than double when cameras were rotated by 45°, the proposed IcoSweepNet maintained stable performance (RMS error of 2.53 vs 5.17 for the baseline).
Computational Efficiency
The method is highly efficient, processing four fisheye images into a 360° depth map in approximately 0.42 to 0.73 seconds on a laptop GPU (NVIDIA RTX 2080 Max-Q). This speed, combined with its robustness, makes it suitable for real-time robotics applications.
Conclusion
The CrownConv architecture and icosahedron-based sweeping provide a robust and efficient framework for omnidirectional depth estimation. By standardizing the input space and leveraging efficient 2D convolutions, the method overcomes the limitations of traditional fisheye processing. Future work will focus on bridging the sim-to-real gap regarding reflective surfaces and further optimizing inference speed.
Summary updated