Abstract
Monocular 360° depth estimation is a critical task for immersive technologies and robotics, yet it is complicated by the inherent "polar distortion" of Equirectangular Projection (ERP). We introduce CRF360D, a novel framework that leverages Spherical Fully-Connected CRFs (SF-CRFs) to effectively model depth by integrating local planar details with global spherical relationships. The core innovation, the Spherical Window Transform (SWT), utilizes the rotational invariance of the sphere to provide a computationally efficient alternative to traditional geometric corrections, operating significantly faster than existing methods. Evaluated on Stanford2D3D, Matterport3D, and Structured3D datasets, CRF360D achieves state-of-the-art accuracy and superior visual clarity.
Introduction
Most 360° images are stored in Equirectangular Projection (ERP), a mapping that projects a spherical surface onto a 2D plane. This projection causes severe distortion at the poles: pixels that are close together on a physical sphere are stretched far apart on the 2D plane. Standard computer vision tools, such as CNNs, struggle with this because their square windows assume a flat world. While a square window works well at the equator, it captures almost no useful spherical context at the poles. Previous solutions attempted to project images onto "tangent patches" to mitigate this, but these methods are computationally heavy and often require slow, pixel-by-pixel geometric lookups.
Methodology
The authors introduce Spherical Fully-Connected CRFs (SF-CRFs) to model depth by balancing local planar details and global spherical relationships. The framework consists of two novel modules:
- Spherical Window Transform (SWT): By recognizing that a sphere looks the same regardless of rotation (rotational invariance), the authors use a "perfect" window from the equator as a template. This template is then rotated to any other position on the globe, allowing the model to identify correct spherical neighbors anywhere on the map with extreme efficiency. This module can process a high-resolution image in just 0.038 seconds on a standard CPU.
- Planar-Spherical Interaction (PSI): This module acts as a bridge, fusing features from standard "flat" windows (ideal for local textures) with features from the transformed "spherical" windows (ideal for global structure).
The overall architecture uses a standard backbone (e.g., EfficientNet) as an encoder and stacks these SF-CRF blocks in the decoder to progressively refine the depth map.
Experimental Results
CRF360D was evaluated on three major indoor 360° benchmarks: Stanford2D3D, Matterport3D, and Structured3D.
Performance and Efficiency
- State-of-the-Art Accuracy: CRF360D outperformed all existing methods across all three datasets. Notably, it achieved a 9.6% improvement in Absolute Relative error over the previous leader, HRDFuse, on the Stanford2D3D dataset.
- Computational Speed: The SWT module is significantly faster than competing geometric corrections. It takes only 0.038s, compared to 0.25s for Cube Maps and 1.057s for Tangent Patches—a roughly 25x speedup over the latter.
- Visual Fidelity: Qualitatively, the depth maps produced by CRF360D show much sharper structural details, such as the thin legs of a chair or distinct box edges, which are often blurred by other 360° models.
Key Insights
- Geometry as a Shortcut: By utilizing spherical rotation instead of expensive per-pixel coordinate math, the authors made the "spherical" view of data nearly as cheap to compute as the "flat" view.
- Dual Perspectives: The PSI module demonstrates that fixing distortion does not require sacrificing local details. Simultaneous planar and spherical processing captures both sharp edges and consistent global layouts.
- Granular Correction: The researchers found that smaller 4x4 windows performed better than standard 8x8 or 7x7 windows, as they allow for more frequent rotations and a more granular correction of distortion.
Conclusion
CRF360D effectively addresses the polar distortion problem in 360° depth estimation with unprecedented efficiency. While the current template window is fixed, future work will explore multi-style templates to further enrich the model's representation and test the framework on real-world robotic platforms for autonomous navigation.
Summary updated