Abstract
This paper investigates how the non-linear distortion characteristic of wide-angle (panomorph) lenses influences the performance of CNN-based depth estimation. By simulating various lens profiles with locally augmented resolution, the authors demonstrate that increased pixel density in specific image regions directly correlates with improved local depth estimation accuracy. Furthermore, the study shows that incorporating manufacturing tolerances into the training process significantly enhances the robustness of these models in real-world applications.
Introduction
Most computer vision models are optimized for standard perspective images, which assume a pinhole camera model with a narrow field of view (FOV). However, wide-angle and super-wide-angle lenses (up to 360° FOV) are becoming ubiquitous in fields such as autonomous driving and surveillance. These lenses introduce severe non-linear barrel distortion. Traditionally, this is addressed by "dewarping" images, which often results in pixel stretching and the loss of peripheral information. This study explores an alternative: treating spatially-varying distortion as a feature to locally augment resolution and improve downstream tasks like single-image depth estimation.
Methodology
To evaluate the impact of distortion without the cost of extensive real-world data collection, the authors utilized a simulation-based approach. They warped 360° panoramic RGB-D images from the SunCG and Matterport3D datasets to simulate four distinct lens profiles: a standard linear fisheye and three "panomorph" lenses designed to increase pixel density in the center, edges, or median zones.
Identical encoder-decoder CNN architectures were trained on these simulated datasets. To account for real-world manufacturing imperfections, a separate "tolerancing" experiment was conducted, where random variations within commercial specs were applied to the distortion profiles during training.
Results and Discussion
The experimental results revealed that while global error rates (RMSE) were similar across all lens profiles, local accuracy was significantly higher in the zones where the specific lens had augmented resolution. This confirms that hardware-software co-design—where optical distortion is tailored to the AI task—can effectively boost performance in critical areas without sacrificing the wide field of view.
Furthermore, the inclusion of manufacturing tolerances in the training data proved vital for robustness. Networks trained with these variations outperformed standard models when tested on data with optical biases, particularly in the affected fields of view. While perspective crops still offer superior absolute accuracy, panomorph lenses successfully bridge the gap by providing high local resolution while preserving the full 180° scene.
Conclusion
This research highlights the potential of leveraging optical distortion as a tool for enhancing CNN performance. By designing lenses that prioritize resolution in task-critical zones and accounting for manufacturing tolerances during training, engineers can develop more robust and accurate wide-angle vision systems. Future work will involve validating these findings with physical panomorph hardware and exploring applications in other computer vision domains.
Summary updated