Abstract
Disparity/depth estimation from sequences of stereo images is an important element in 3D vision. Owing to occlusions, imperfect settings and homogeneous luminance, accurate estimate of depth remains a challenging problem. Targetting view synthesis, we propose a novel learning-based framework making use of dilated convolution, densely connected convolutional modules, compact decoder and skip connections. The network is shallow but dense, so it is fast and accurate. Two additional contributions—a non-linear adjustment of the depth resolution and the introduction of a projection loss—lead to reduction of estimation error by up to 20% and 25% respectively. The results show that our network outperforms state-of-the-art methods with an average improvement in accuracy of depth estimation and view synthesis by approximately 45% and 34% respectively. Where our method generates comparable quality of estimated depth, it performs 10 times faster than those methods.
Introduction
Accurate depth estimation is a cornerstone of view synthesis, the process of generating novel perspectives from existing image data. While deep learning has significantly advanced the field, high-performance models often suffer from extreme computational overhead, making them impractical for real-time applications such as augmented reality or autonomous navigation. Furthermore, standard depth estimation models often fail to prioritize the foreground, where depth errors are most perceptually jarring in synthesized views. This paper introduces a shallow yet dense convolutional neural network (CNN) designed for high-speed, high-accuracy depth estimation specifically optimized for view synthesis.
Methodology: Shallow but Dense Architecture
The proposed framework achieves efficiency by utilizing a siamese-like feature extractor that processes input images at a reduced resolution (downsampled by a factor of 8). To maintain high accuracy despite the lower resolution, the architecture incorporates several advanced components:
- Dense Blocks: By connecting layers to every subsequent layer, the network maintains high information flow within a relatively shallow structure.
- Dilated Convolutions: These allow the network to expand its receptive field and capture large disparities without the computational cost of larger filters.
- Compact Decoder: A step-wise upsampling mechanism with skip connections is employed to recover fine spatial details lost during initial downsampling.
Technical Innovations
Non-linear Depth Adjustment
A primary innovation of this work is the application of an exponential adjustment to the ground truth depth during training ($z' = \hat{z}^p$). By setting $p > 1$ (e.g., $p=1.5$), the network effectively "stretches" the resolution for objects closer to the camera. This forces the model to prioritize precision in the near-field, which is critical for high-quality view synthesis.
Projection Loss
To enforce geometric consistency, the model incorporates a projection loss. This loss function evaluates the network's ability to reconstruct the left image by warping the right image using the estimated depth map. This self-consistency check provides a powerful geometric constraint that helps the model learn the underlying structure of the scene beyond simple pixel matching.
Results and Discussion
The model was evaluated on standard benchmarks (Sintel, Driving, Monkaa) and a custom synthetic dataset of 25,000 stereo pairs generated in the Unity game engine.
- Computational Efficiency: The network processes a stereo pair in approximately 0.21 seconds on an NVIDIA Tesla M60, representing a 10-fold speedup over state-of-the-art models like PSMNet.
- Accuracy Gains: Ablation studies demonstrate that the non-linear depth adjustment and projection loss reduce estimation error by 20% and 25%, respectively.
- Visual Fidelity: The method produces cleaner edges and fewer artifacts in synthesized views compared to traditional disparity-focused models, particularly in complex, narrow structures.
Conclusion
By combining a dense, shallow architecture with perceptually-motivated depth adjustment and geometric loss constraints, we have developed a depth estimation framework that is both fast and accurate. The 10x speed improvement over existing methods makes this approach highly suitable for real-time 3D vision tasks where view synthesis quality is paramount.
Summary updated