Introduction
Monocular depth estimation (MDE) is a fundamental task for spatial awareness in robotics and autonomous systems. While deep learning has significantly advanced MDE performance, a stark divide exists between high-accuracy models and those suitable for real-time deployment on edge devices. State-of-the-art models often exceed 200 million parameters, while lightweight alternatives frequently sacrifice critical precision. This paper introduces an ultra-lightweight MDE network that bridges this gap using Knowledge Distillation (KD) and strategic auxiliary data utilization.
Lightweight Architecture Design
The core of the proposed solution is a highly efficient encoder-decoder architecture designed to minimize computational overhead without losing spatial resolution.
The model utilizes a MobileNet-v2 backbone as the encoder. To maintain a low parameter count, the researchers implemented a novel feature compression strategy. Feature maps are extracted at four different scales and passed through channel-wise attention modules (denoted as 'G' in the architecture). These maps are then compressed to a fixed width of only 16 channels using 5x5 convolutional layers. By upsampling and concatenating these multi-scale features, the decoder retains rich spatial information while operating with a fraction of the parameters found in standard U-Net or ResNet-based decoders.
Overcoming the Capacity Gap via Knowledge Distillation
Knowledge Distillation typically involves a large "teacher" model guiding a smaller "student." However, a significant "capacity gap" often prevents the student from effectively learning the teacher's complex mappings.
To address this, the authors leverage auxiliary unlabeled data. By using a ResNet-34 teacher to generate pseudo-labels for additional datasets like ScanNet, the student is exposed to a broader data manifold. This process provides more dense signals for the student to approximate the teacher's behavior. A key finding, termed the "Similar Scale" Hypothesis, suggests that this distillation is most effective when the auxiliary data shares a similar depth distribution and scene scale with the target domain (e.g., using indoor ScanNet data to improve performance on the indoor NYU-v2 dataset).
Key Results and Performance
The resulting model demonstrates an exceptional balance between efficiency and accuracy. With only 1.7 million parameters, the student model achieves performance levels previously reserved for models 100 times its size.
As shown in the performance comparison, the proposed method (Ours) occupies the top-left quadrant of the accuracy-parameter trade-off, outperforming established lightweight models like Wofk et al. and Nekrasov et al. while rivaling the accuracy of much heavier architectures like Hu et al. and Lee et al.
In practical terms, the model achieves an inference speed of 11ms on a standard desktop GPU and shows robust generalization across different indoor datasets (SUNRGBD, TUM), confirming that the KD process helps the model learn fundamental geometric features rather than dataset-specific noise.
Conclusion
By combining a parameter-efficient architecture with a data-driven knowledge distillation strategy, this research proves that high-quality depth estimation does not require massive computational resources. The use of auxiliary data to bridge the capacity gap provides a scalable path for developing high-performance, real-time vision models for mobile robotics and edge computing.
Summary updated