TL;DR
This paper introduces a straightforward, CNN-based framework for 3D occupancy estimation in autonomous driving, focusing on network design, optimization, and evaluation. It proposes a novel distance-based metric for occupancy evaluation and establishes a benchmark for depth estimation, demonstrating competitive performance against existing methods.
What Problem Does This Paper Address?
Autonomous driving systems need a comprehensive understanding of their 3D environment for safe navigation. While Bird's Eye View (BEV) perception tasks like 3D detection and map segmentation have advanced, they often overlook crucial 3D geometric information. Reconstructing the precise 3D geometry of driving scenes is therefore a critical, yet challenging, task. Existing methods like monocular depth estimation provide pixel-level depth but require complex post-processing to form 3D representations, which can be inconsistent. This paper aims to simplify and improve 3D occupancy estimation by proposing a unified framework that also explores its relationship with depth estimation and 3D reconstruction.
Methods
The proposed framework, "SimpleOccupancy," is a CNN-based approach that takes surrounding-view images as input. Key methodological aspects include:
- Network Design: The architecture is inspired by stereo matching, utilizing a 2D encoder (ResNet) for feature extraction followed by a parameter-free interpolation method (similar to Simple-BEV) to project features into a 3D volume. A 3D CNN then aggregates these features.
- Volume Rendering for Training: The authors leverage volume rendering, inspired by NeRF, to generate depth maps for supervised training. This allows for supervision using ground truth depth maps.
- Occupancy Probability and SDF: The framework can output either occupancy probability (via Sigmoid) or Signed Distance Function (SDF) values, enabling both occupancy estimation and 3D reconstruction.
- Novel Evaluation Metric: A "discrete depth metric" is proposed, which is a distance-based metric that evaluates predictions based on the distance between the first predicted occupancy point and the ground truth point cloud along a ray. This is argued to be more equitable than classification metrics, especially for sparse ground truth data.
- Optimization Strategies: Both supervised learning (using depth loss, BCE loss, or L1 loss) and self-supervised learning (using photometric consistency loss) are explored.
Datasets
- DDAD [16]: Used for training and evaluation, particularly for supervised learning and depth estimation benchmarks. It provides dense ground truth depth maps.
- Nuscenes [3]: Used for self-supervised learning experiments and depth estimation benchmarks.
The paper does not introduce new datasets but focuses on evaluating existing ones with novel metrics and approaches.
Key Insights and Ideas
- Simplicity is Key: The framework demonstrates that a simple, yet well-designed, CNN architecture can achieve competitive results in 3D occupancy estimation, highlighting the importance of network design and optimization.
- Volume Rendering as a Unifying Tool: Using volume rendering for depth map generation proves effective for supervised training and allows for a unified approach to depth estimation, occupancy estimation, and 3D reconstruction.
- Discrete Depth Metric Superiority: The proposed discrete depth metric offers a more robust and fair evaluation of 3D occupancy compared to traditional classification metrics, especially in scenarios with sparse ground truth.
- Point-Level Pretraining Benefits: The authors show that their point-level training strategy can serve as an effective pretraining step for voxel-level semantic occupancy prediction methods, improving their performance.
- SDF for Reconstruction, Density for Occupancy: While both SDF and density representations can be used, SDF appears to yield better results for mesh-based 3D reconstruction, whereas density might be more flexible for occupancy estimation.
Key Results
- Competitive Depth Estimation: The proposed method achieves competitive performance in depth estimation benchmarks on both DDAD and Nuscenes datasets, often matching or exceeding state-of-the-art monocular depth estimation methods. For example, on Nuscenes (supervised), it achieves an Abs Rel of 0.542 and an RMSE log of 0.179.
- Effective Occupancy Evaluation: The discrete depth metric demonstrates its efficacy in evaluating occupancy predictions, showing better discrimination between good and bad predictions compared to classification metrics in ablation studies.
- Improved Semantic Occupancy: Pretraining with the proposed point-level sampling strategy improves the performance of a voxel-level semantic occupancy method (SurroundOcc) by 1-2% in SC IoU.
- Inference Time: While competitive in accuracy, the inference time is higher than monocular depth estimation methods, with the 3D CNN and rendering components being the main contributors.
Limitations and Future Work
- Sequence Information: The current framework does not incorporate temporal information from sequences, which is a promising direction for improving performance.
- Voxel Resolution: The current voxel resolution (0.4m) is considered coarse. Future work could explore higher resolutions (e.g., 0.2m) for better geometric detail.
- MLPs for Geometry Regression: The authors suggest using MLPs to regress final geometry attributes, which could allow for additional regularization losses like eikonal loss.
- Computational Cost: The 3D convolution and rendering steps contribute to a higher inference time compared to monocular methods.
- SDF vs. Density Trade-offs: While SDF is better for reconstruction, density might be more flexible for occupancy. Further exploration of these trade-offs is warranted.
Summary updated