Towards real-time unsupervised monocular depth estimation on CPU

Matteo Poggi, Filippo Aleotti, Fabio Tosi, Stefano Mattoccia

Abstract

PyD-Net is a lightweight, pyramidal convolutional neural network designed for unsupervised monocular depth estimation on low-power CPUs. By employing a coarse-to-fine architecture with 94% fewer parameters than state-of-the-art models, it enables real-time depth sensing on embedded devices like the Raspberry Pi without requiring high-end GPUs.

Introduction

Estimating depth from a single image (monocular depth estimation) is a critical capability for robotics, augmented reality, and autonomous navigation. While deep learning has significantly advanced this field, existing models are typically massive, "black-box" architectures that require high-end, power-hungry GPUs to operate in real-time. This creates a significant barrier for deployment on small drones (UAVs), wearable assistive devices, and vacuum robots that operate under strict power and hardware constraints. PyD-Net addresses this gap by providing an efficient network capable of running on standard CPUs and embedded ARM processors while maintaining high accuracy.

The PyD-Net Architecture

PyD-Net (Pyramidal Depth Network) departs from traditional, heavy "U-Net" style architectures in favor of a pyramidal approach. The design focuses on extreme parameter efficiency without sacrificing the global context necessary for depth perception.

PyD-Net architecture showing the pyramidal feature extractor and scale-specific decoders (L1-L6).

Pyramidal Feature Extractor

Instead of a single deep bottleneck, the network utilizes a small encoder (12 layers) to create a pyramid of features at six different scales, ranging from 1/2 to 1/64 of the original resolution.

Coarse-to-Fine Decoders

At each level of the pyramid, a shallow decoder infers a depth map. This strategy allows the model to capture broad environmental structures at low resolutions and progressively refine sharp details at higher resolutions. By capping the output at half-resolution and upsampling, the authors achieved a "sweet spot" for performance on embedded hardware.

Unsupervised Training Paradigm

The model is trained without expensive ground-truth depth labels (such as LiDAR). Instead, it utilizes stereo image pairs during training to learn the "disparity" (pixel shift) required to reconstruct the left image from the right image. The training is guided by a composite loss function including image reconstruction loss (SSIM and L1 distance), a smoothness term to reduce noise, and a left-right consistency check. Once trained, the network requires only a single image for inference.

Experimental Results

PyD-Net achieves a massive reduction in complexity, using just 1.9 million parameters—a 94% reduction compared to state-of-the-art models like Monodepth.

Qualitative depth estimation results showing the input scene and corresponding depth maps at different scales.

Performance and Accuracy

  • Speed: On a standard i7-6700K CPU, PyD-Net runs at over 8 Hz (up to 40 Hz in optimized configurations).
  • Embedded Deployment: On a Raspberry Pi 3, the model generates a depth map in approximately 1.7 seconds while consuming only ~3.5W.
  • Accuracy: Despite its size, PyD-Net remains highly competitive with models 16x larger and outperforms several older supervised methods.
  • Memory: The model requires less than 150 MB of RAM, making it viable for memory-constrained devices.

Conclusion

PyD-Net demonstrates that high-quality monocular depth estimation does not strictly require massive parameter counts or GPU acceleration. By leveraging a pyramidal, coarse-to-fine strategy, real-time depth sensing becomes accessible for low-power embedded systems, opening new possibilities for mobile robotics and edge computing.

Summary updated

Abstract

Unsupervised depth estimation from a single image is a very attractive technique with several implications in robotic, autonomous navigation, augmented reality and so on. This topic represents a very challenging task and the advent of deep learning enabled to tackle this problem with excellent results. However, these architectures are extremely deep and complex. Thus, real-time performance can be achieved only by leveraging power-hungry GPUs that do not allow to infer depth maps in application fields characterized by low-power constraints. To tackle this issue, in this paper we propose a novel architecture capable to quickly infer an accurate depth map on a CPU, even of an embedded system, using a pyramid of features extracted from a single input image. Similarly to state-of-the-art, we train our network in an unsupervised manner casting depth estimation as an image reconstruction problem. Extensive experimental results on the KITTI dataset show that compared to the top performing approach our network has similar accuracy but a much lower complexity (about 6% of parameters) enabling to infer a depth map for a KITTI image in about 1.7 s on the Raspberry Pi 3 and at more than 8 Hz on a standard CPU. Moreover, by trading accuracy for efficiency, our network allows to infer maps at about 2 Hz and 40 Hz respectively, still being more accurate than most state-of-the-art slower methods. To the best of our knowledge, it is the first method enabling such performance on CPUs paving the way for effective deployment of unsupervised monocular depth estimation even on embedded systems.
BibTeX
@inproceedings{poggi2018towards,
  author = {Matteo Poggi and Filippo Aleotti and Fabio Tosi and Stefano Mattoccia},
  title = {{Towards real-time unsupervised monocular depth estimation on CPU}},
  booktitle = {IEEE/RSJ International Conference on Intelligent Robots and Systems},
  year = {2018},
  doi = {10.1109/iros.2018.8593814},
}
Figure