Evaluating the Impact of Wide-Angle Lens Distortion on Learning-Based Depth Estimation

Julie Buquet, Jinsong Zhang, Patrice Roulet, Simon Thibault, Jean-François LalondeWorkshops @ CVPR 2021 · DOI

Abstract

This paper investigates how the non-linear distortion characteristic of wide-angle (panomorph) lenses influences the performance of CNN-based depth estimation. By simulating various lens profiles with locally augmented resolution, the authors demonstrate that increased pixel density in specific image regions directly correlates with improved local depth estimation accuracy. Furthermore, the study shows that incorporating manufacturing tolerances into the training process significantly enhances the robustness of these models in real-world applications.

Introduction

Most computer vision models are optimized for standard perspective images, which assume a pinhole camera model with a narrow field of view (FOV). However, wide-angle and super-wide-angle lenses (up to 360° FOV) are becoming ubiquitous in fields such as autonomous driving and surveillance. These lenses introduce severe non-linear barrel distortion. Traditionally, this is addressed by "dewarping" images, which often results in pixel stretching and the loss of peripheral information. This study explores an alternative: treating spatially-varying distortion as a feature to locally augment resolution and improve downstream tasks like single-image depth estimation.

Methodology

To evaluate the impact of distortion without the cost of extensive real-world data collection, the authors utilized a simulation-based approach. They warped 360° panoramic RGB-D images from the SunCG and Matterport3D datasets to simulate four distinct lens profiles: a standard linear fisheye and three "panomorph" lenses designed to increase pixel density in the center, edges, or median zones.

Radial distortion profiles for the four simulated lens models (Linear, Center, Edges, Median).

Identical encoder-decoder CNN architectures were trained on these simulated datasets. To account for real-world manufacturing imperfections, a separate "tolerancing" experiment was conducted, where random variations within commercial specs were applied to the distortion profiles during training.

Results and Discussion

The experimental results revealed that while global error rates (RMSE) were similar across all lens profiles, local accuracy was significantly higher in the zones where the specific lens had augmented resolution. This confirms that hardware-software co-design—where optical distortion is tailored to the AI task—can effectively boost performance in critical areas without sacrificing the wide field of view.

Visual comparison of RGB and depth map outputs across different lens distortion profiles.

Furthermore, the inclusion of manufacturing tolerances in the training data proved vital for robustness. Networks trained with these variations outperformed standard models when tested on data with optical biases, particularly in the affected fields of view. While perspective crops still offer superior absolute accuracy, panomorph lenses successfully bridge the gap by providing high local resolution while preserving the full 180° scene.

Conclusion

This research highlights the potential of leveraging optical distortion as a tool for enhancing CNN performance. By designing lenses that prioritize resolution in task-critical zones and accounting for manufacturing tolerances during training, engineers can develop more robust and accurate wide-angle vision systems. Future work will involve validating these findings with physical panomorph hardware and exploring applications in other computer vision domains.

Summary updated

BibTeX
@inproceedings{buquet2021evaluating,
  author = {Julie Buquet and Jinsong Zhang and Patrice Roulet and Simon Thibault and Jean-François Lalonde},
  title = {{Evaluating the Impact of Wide-Angle Lens Distortion on Learning-Based Depth Estimation}},
  booktitle = {Workshops @ CVPR},
  year = {2021},
  doi = {10.1109/cvprw53098.2021.00409},
}
Figure