A Simple Baseline for Supervised Surround-view Depth Estimation

Xianda Guo, Wenjie Yuan, Yunpeng Zhang, Tian Yang, Chenming Zhang, Zheng Zhu

Abstract

Depth estimation is a fundamental step in 3D perception for autonomous driving. While monocular depth estimation has progressed significantly, most research focuses on front-view cameras, ignoring correlations across surround-view cameras. We propose S3Depth, a simple yet effective supervised baseline for surround-view depth estimation. By employing a global-to-local feature extraction module and a novel Adjacent-view Attention mechanism, our model jointly predicts depth maps across multiple cameras, achieving state-of-the-art performance on the nuScenes and DDAD benchmarks.

1. Introduction

Modern autonomous vehicles utilize a suite of cameras to provide a 360° surround-view. Standard monocular methods treat each camera independently, failing to exploit spatial correlations. Conversely, Multi-View Stereo (MVS) techniques struggle with surround-view setups because the cameras typically have very small overlapping regions. While self-supervised surround-view methods have been explored, their accuracy often falls short of safety-critical requirements. This paper introduces a robust supervised baseline, S3Depth (also referred to as AVT-SSDepth), that explicitly models inter-view dependencies at shared boundaries.

2. Methodology

The proposed architecture consists of three primary components designed to capture both intra-view and inter-view information.

2.1 Global-to-Local Feature Extractor

Instead of a pure CNN or Transformer, the model uses a hybrid backbone (specifically MPViT). This allows the network to capture fine-grained local textures via CNN layers while maintaining a global understanding of the scene via Transformer layers. This hybrid approach is essential as depth estimation requires both local edge detection and global geometric reasoning.

2.2 Adjacent-View Attention Module

The core innovation is the Adjacent-View Attention mechanism. Rather than performing computationally expensive attention across all six cameras at once, the model uses an alternating attention strategy:

  • Self-Attention: Computes features within a single camera view to capture intra-view context.
  • Adjacent-Attention: Computes attention between a "current" view and its immediate left and right neighbors. This allows the model to "stitch" features across the small overlapping regions, ensuring consistency across the entire 360° suite.
The Adjacent-View Attention module enables feature propagation within and across neighboring camera views.

2.3 Depth Head and Training

A decoder upsamples these multi-scale, cross-view features to predict high-resolution depth maps for all cameras simultaneously. The model is trained using a combination of a standard L1 loss, comparing predictions to LiDAR ground truth, and a smoothness loss to ensure the depth maps are spatially coherent and noise-free.

3. Experiments and Results

The model was evaluated on the nuScenes and DDAD datasets, which provide 360° coverage and LiDAR ground truth.

3.1 Quantitative Performance

S3Depth sets a new state-of-the-art for supervised surround-view depth. On the nuScenes dataset, it achieved an Absolute Relative Error (Abs Rel) of 0.067, a significant improvement over previous supervised baselines like PixelFormer (0.099) and self-supervised methods like SurroundDepth (0.158).

Quantitative results on the nuScenes dataset demonstrate that S3Depth significantly outperforms existing supervised and self-supervised methods.

3.2 Key Insights

  • Efficiency of Adjacency: Focusing on neighboring views provides sufficient cross-view interaction to stabilize depth predictions at boundaries without the overhead of full 360° attention.
  • Convergence Speed: The Adjacent-view Attention module facilitates faster training convergence compared to models lacking cross-view interaction.
  • Robustness: The model demonstrates strong cross-dataset generalization, outperforming specialized models even when trained on different sensor configurations (e.g., training on DDAD and testing on nuScenes).

4. Conclusion

S3Depth provides a strong supervised baseline for surround-view depth estimation. By leveraging adjacent-view attention, the model effectively fuses features across cameras with minimal overlap. Future work will focus on integrating temporal consistency and exploring the impact of high-quality depth on downstream Bird's-Eye-View (BEV) perception tasks.

Summary updated

Abstract

Depth estimation has been widely studied and serves as the fundamental step of 3D perception for autonomous driving. Though significant progress has been made for monocular depth estimation in the past decades, these attempts are mainly conducted on the KITTI benchmark with only front-view cameras, which ignores the correlations across surround-view cameras. In this paper, we propose S3Depth, a Simple Baseline for Supervised Surround-view Depth Estimation, to jointly predict the depth maps across multiple surrounding cameras. Specifically, we employ a global-to-local feature extraction module which combines CNN with transformer layers for enriched representations. Further, the Adjacent-view Attention mechanism is proposed to enable the intra-view and inter-view feature propagation. The former is achieved by the self-attention module within each view, while the latter is realized by the adjacent attention module, which computes the attention across multi-cameras to exchange the multi-scale representations across surround-view feature maps. Extensive experiments show that our method achieves superior performance over existing state-of-the-art methods on both DDAD and nuScenes datasets.
BibTeX
@inproceedings{guo2025simple,
  author = {Xianda Guo and Wenjie Yuan and Yunpeng Zhang and Tian Yang and Chenming Zhang and Zheng Zhu},
  title = {{A Simple Baseline for Supervised Surround-view Depth Estimation}},
  booktitle = {IEEE/RSJ International Conference on Intelligent Robots and Systems},
  year = {2025},
  doi = {10.1109/iros60139.2025.11246758},
}
Figure