Introduction
Wide-angle and fish-eye lenses are essential for capturing expansive scenes in applications like autonomous driving and surveillance. However, they introduce significant "barrel distortion," where straight lines appear curved. While traditional feature-based methods struggle in environments lacking clear lines, recent deep learning approaches often fail to capture the complex, non-linear mapping required for high-quality rectification. This paper addresses these challenges by introducing a hierarchical framework guided by a multi-level curriculum.
Hierarchical Rectification Framework
The core of the proposed approach is a Multi-Level Curriculum that breaks down the rectification task into three progressive stages, analogous to a human artist's workflow: starting with a structural sketch, adding semantic coloring, and finally rendering fine textures.
The framework consists of specialized modules:
- Structure Recovery: Focuses on fixing the geometric skeleton and contours.
- Semantics Embedding: Corrects the shapes and boundaries of high-level objects.
- Texture Rendering: Produces the final high-resolution RGB image.
To handle the spatial misalignment between distorted inputs and rectified outputs, the authors introduce the Location-Aligned Connection (LAC). Unlike standard skip connections in U-Nets, LAC "revises" encoder feature maps to align with the decoder's geometry, ensuring more accurate feature fusion.
Distortion-Aware Pre-training
Standard pre-training on datasets like ImageNet is often suboptimal for geometric tasks because it focuses on classification rather than spatial transformations. The authors propose a Distortion-Aware Pre-training strategy. Instead of predicting obscure parametric labels, the model is trained to estimate a visually observable "distortion level" ($D_{max}$).
This pre-training phase teaches the encoders to perceive the severity of the warp across different levels of abstraction (structure, semantics, and texture) before the rectification training begins.
Experimental Results
The model was trained on a synthetic dataset derived from ADE20K and evaluated across multiple domains, including COCO, Place365, and real-world footage from GoPro and other wide-angle cameras.
The proposed method achieved a PSNR of 26.71 and an SSIM of 0.88, significantly outperforming the previous state-of-the-art (Liao et al. 2020), which achieved 23.02 and 0.71, respectively. Qualitatively, the model excels at restoring straight lines and maintaining natural global scene distributions, even in complex real-world scenarios without specific camera calibration.
Conclusion
By adopting a "geometry-first" approach through a multi-level curriculum and distortion-aware pre-training, this work sets a new benchmark for barrel distortion rectification. While the model's reliance on semantic segmentation presents challenges for cross-domain generalization, it provides a robust foundation for future research in spatially-aware image restoration.
Summary updated