Abstract
Depth estimation is a fundamental step in 3D perception for autonomous driving. While monocular depth estimation has progressed significantly, most research focuses on front-view cameras, ignoring correlations across surround-view cameras. We propose S3Depth, a simple yet effective supervised baseline for surround-view depth estimation. By employing a global-to-local feature extraction module and a novel Adjacent-view Attention mechanism, our model jointly predicts depth maps across multiple cameras, achieving state-of-the-art performance on the nuScenes and DDAD benchmarks.
1. Introduction
Modern autonomous vehicles utilize a suite of cameras to provide a 360° surround-view. Standard monocular methods treat each camera independently, failing to exploit spatial correlations. Conversely, Multi-View Stereo (MVS) techniques struggle with surround-view setups because the cameras typically have very small overlapping regions. While self-supervised surround-view methods have been explored, their accuracy often falls short of safety-critical requirements. This paper introduces a robust supervised baseline, S3Depth (also referred to as AVT-SSDepth), that explicitly models inter-view dependencies at shared boundaries.
2. Methodology
The proposed architecture consists of three primary components designed to capture both intra-view and inter-view information.
2.1 Global-to-Local Feature Extractor
Instead of a pure CNN or Transformer, the model uses a hybrid backbone (specifically MPViT). This allows the network to capture fine-grained local textures via CNN layers while maintaining a global understanding of the scene via Transformer layers. This hybrid approach is essential as depth estimation requires both local edge detection and global geometric reasoning.
2.2 Adjacent-View Attention Module
The core innovation is the Adjacent-View Attention mechanism. Rather than performing computationally expensive attention across all six cameras at once, the model uses an alternating attention strategy:
- Self-Attention: Computes features within a single camera view to capture intra-view context.
- Adjacent-Attention: Computes attention between a "current" view and its immediate left and right neighbors. This allows the model to "stitch" features across the small overlapping regions, ensuring consistency across the entire 360° suite.
2.3 Depth Head and Training
A decoder upsamples these multi-scale, cross-view features to predict high-resolution depth maps for all cameras simultaneously. The model is trained using a combination of a standard L1 loss, comparing predictions to LiDAR ground truth, and a smoothness loss to ensure the depth maps are spatially coherent and noise-free.
3. Experiments and Results
The model was evaluated on the nuScenes and DDAD datasets, which provide 360° coverage and LiDAR ground truth.
3.1 Quantitative Performance
S3Depth sets a new state-of-the-art for supervised surround-view depth. On the nuScenes dataset, it achieved an Absolute Relative Error (Abs Rel) of 0.067, a significant improvement over previous supervised baselines like PixelFormer (0.099) and self-supervised methods like SurroundDepth (0.158).
3.2 Key Insights
- Efficiency of Adjacency: Focusing on neighboring views provides sufficient cross-view interaction to stabilize depth predictions at boundaries without the overhead of full 360° attention.
- Convergence Speed: The Adjacent-view Attention module facilitates faster training convergence compared to models lacking cross-view interaction.
- Robustness: The model demonstrates strong cross-dataset generalization, outperforming specialized models even when trained on different sensor configurations (e.g., training on DDAD and testing on nuScenes).
4. Conclusion
S3Depth provides a strong supervised baseline for surround-view depth estimation. By leveraging adjacent-view attention, the model effectively fuses features across cameras with minimal overlap. Future work will focus on integrating temporal consistency and exploring the impact of high-quality depth on downstream Bird's-Eye-View (BEV) perception tasks.
Summary updated