Abstract
Depth estimation is a critical task for computer vision applications like autonomous driving. While self-supervised approaches—which rely on camera motion rather than ground truth depth maps—have advanced significantly, most still operate on a single-frame basis. This work explores integrating recurrent blocks and attention mechanisms to exploit temporal information from previous frames. By proposing new neural network architectures for monocular depth estimation, the authors demonstrate on the KITTI dataset that these modifications effectively leverage temporal context to improve prediction quality.
Introduction
Estimating 3D depth from a single 2D image is an inherently ill-posed problem, as multiple 3D configurations can result in the same 2D projection. Traditional self-supervised methods address this by comparing adjacent video frames to learn geometry. However, by predicting depth frame-by-frame, these models ignore the rich temporal "memory" available in video sequences.
This paper introduces a novel pipeline that incorporates ConvGRU recurrent blocks and self-attention mechanisms. By maintaining a hidden state of the scene's geometry over time, the model produces more consistent and accurate depth maps in real-time, effectively bridging the gap between independent frame analysis and continuous temporal understanding.
Methodology: Beyond Single-Frame Estimation
The authors build upon the Monodepth2 framework, a standard in self-supervised learning that uses photometric consistency between a target frame and source frames to supervise the network.
To enhance this baseline, the following architectural modifications were introduced:
- Recurrent ConvGRU Blocks: Convolutional Gated Recurrent Units are integrated into the decoder at four different scales. These blocks pass hidden states from previous timestamps to the current one, allowing the network to "remember" spatial features.
- Self-Attention Fusion: A dedicated Fusion block combines current frame features with temporal hidden states. The authors utilize Residual Self-Attention units to capture long-range spatial dependencies, which helps the model understand global scene structure better than standard local convolutions.
- Two-Stage Training: The complexity of recurrent training is managed via a two-stage strategy: initial training with zero-initialized hidden states, followed by refinement using learned temporal context.
Experimental Results
The models were evaluated on the KITTI dataset (Eigen split), the industry standard for autonomous driving benchmarks.
Performance and Efficiency
The proposed "Model #2" (incorporating ConvGRU and Self-Attention Fusion) achieved superior results in Root Mean Square Error (RMSE) compared to the Monodepth2 baseline. Crucially, the model maintains high efficiency, running at 124 ± 16 FPS on an RTX6000 GPU. This makes it significantly more practical for real-world robotics than heavy supervised models like DORN, which often run at much lower frame rates (approx. 15 FPS).
Qualitative Improvements
Visual analysis indicates that the recurrent approach is particularly effective at:
- Moving Objects: Better depth prediction for cyclists and cars.
- Distant Regions: Reduced variance in far-off areas where single-frame models often struggle.
- Object Boundaries: Sharper, more accurate edges for complex or angular shapes.
Discussion and Future Work
The study confirms that temporal consistency is vital for robust depth estimation. By aggregating history over a 10-frame window, the model resolves ambiguities caused by lighting changes or sensor noise that typically trip up single-frame models.
However, limitations remain. The model is constrained by GPU memory to 12-frame sequences, and like most self-supervised methods, it struggles with the "static camera" problem where moving objects violate the basic ego-motion assumption. Future research may involve integrating cost volumes or feature-based losses to further narrow the performance gap with fully supervised LiDAR-based systems.
Summary updated