IEBins: Iterative Elastic Bins for Monocular Depth Estimation

Shuwei Shao, Zhongcai Pei, Xingming Wu, Zhong Liu, Weihai Chen, Zhengguo Li

Abstract

Monocular Depth Estimation (MDE) is a fundamental yet ill-posed challenge in computer vision, requiring the prediction of 3D depth from a single 2D image. Traditional classification-regression methods often struggle with a trade-off between computational complexity and precision. This paper introduces IEBins (Iterative Elastic Bins), a novel framework that employs an iterative search process with "elastic" bins. By dynamically adjusting search ranges based on model uncertainty, IEBins avoids the error accumulation typical of iterative refinement while achieving state-of-the-art accuracy and significantly higher inference speeds than previous top-tier models.

Introduction

Predicting depth from a single viewpoint is essential for applications ranging from autonomous driving to augmented reality. However, because a single 2D image can represent an infinite number of 3D configurations, MDE remains a complex problem. Current high-performing models typically discretize the depth range into "bins" (intervals), predicting bin membership before calculating a final depth value.

While effective, these methods face a dilemma: using a high number of bins (e.g., 256) is computationally expensive, whereas iterative refinement—starting with coarse bins and narrowing down—is prone to "locked-in" errors. If an initial coarse prediction is slightly incorrect, the model often cannot recover in subsequent stages. IEBins addresses this by introducing elasticity into the binning process, allowing the model to "zoom in" on depth ranges while maintaining the flexibility to correct early errors.

Methodology

1. Iterative Search with Small Bin Counts

Rather than processing hundreds of bins simultaneously, IEBins utilizes a small, fixed number of bins (typically 16) across multiple stages. The process begins with a coarse search across the entire depth range. In each subsequent iteration, the model identifies a "target bin"—the most likely depth interval—and subdivides only that specific bin into 16 smaller sub-bins for the next stage. This "zoom-in" approach allows for high precision without the memory overhead of massive bin arrays.

2. Elastic Target Bins and Uncertainty

To solve the problem of error accumulation, IEBins introduces elasticity. The system calculates the uncertainty of each prediction based on the variance of the probability distribution. If the model exhibits high uncertainty, the target bin is elastically widened. This ensures that the ground-truth depth remains within the search range for the next iteration, even if the initial prediction was imprecise. This mechanism allows the model to recover from potential errors that would cause traditional iterative models to fail.

3. Architecture

The IEBins architecture leverages a Swin-Transformer backbone for robust feature extraction. The iterative optimization is handled by a Gated Recurrent Unit (GRU). While GRUs are traditionally used for temporal sequences, IEBins uses them to model the evolution of a depth prediction. The GRU maintains a "hidden state" that carries global scene context across iterations, allowing the model to refine local depth details while staying consistent with the overall image structure.

Experimental Results

Benchmarks and Accuracy

IEBins was evaluated on the industry-standard KITTI (outdoor) and NYU-Depth-v2 (indoor) datasets. On the KITTI benchmark, IEBins achieved an Absolute Relative Error (Abs Rel) of 0.050, outperforming established leaders such as PixelFormer and NeWCRFs. Qualitatively, the model excels at capturing thin structures—such as poles and human silhouettes—that are frequently blurred or "smeared" by other depth estimation frameworks.

Efficiency and Speed

Despite its iterative nature, IEBins is highly efficient. In comparative tests, it demonstrated an inference time of 85ms, making it approximately 60% faster than BinsFormer (216ms), despite having a similar parameter count (~273M). This efficiency makes it a strong candidate for real-time applications.

Zero-Shot Generalization

To test robustness, the model was subjected to zero-shot generalization tests on the SUN RGB-D dataset. After being trained on NYU-Depth-v2, IEBins was tested on the unfamiliar SUN RGB-D environments without further fine-tuning. The results showed superior robustness, suggesting that the model learns general geometric principles rather than simply memorizing dataset-specific features.

Discussion and Limitations

While IEBins represents a significant step forward, the authors note two primary areas for improvement:

  1. Boundary Blurring: Because the final depth is calculated as a weighted average of bin centers, object boundaries can occasionally appear slightly blurred.
  2. Supervision Strategy: The model currently uses a loss function based on the final depth value. The authors suggest that direct supervision of the probability distribution might produce even more optimal, "sharper" predictions.

Conclusion

IEBins redefines the classification-regression approach to monocular depth estimation. By combining iterative search with uncertainty-driven elastic bins, the framework achieves a new balance of precision, speed, and robustness. Future work will focus on integrating these capabilities into real-time robotics and autonomous systems where low latency and high accuracy are critical.

Summary updated

BibTeX
@inproceedings{shao2023iebins,
  author = {Shuwei Shao and Zhongcai Pei and Xingming Wu and Zhong Liu and Weihai Chen and Zhengguo Li},
  title = {{IEBins: Iterative Elastic Bins for Monocular Depth Estimation}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2023},
  url = {https://openreview.net/forum?id=jOuxQGRVoQ},
}
Figure